I have two weblogic domains each one has one managed server, the problem is that every 3 or 4 hours may be less than the four process are killed suddenly and in domain console i found that.
./startWebLogic.sh: line 175: 53875 Killed ${JAVA_HOME}/bin/java ${JAVA_VM} ${MEM_ARGS} -Dweblogic.Name=${SERVER_NAME} -Djava.security.policy=${WL_HOME}/server/lib/weblogic.policy ${JAVA_OPTIONS} ${PROXY_SETTINGS} ${SERVER_CLASS}
There is no problem in free memory in server.
free memory
Two possible explanations for this message are the Linux OOM killer and the WebLogic node manager.
You should be able to find evidence for the first in /var/log/messages (grep -i -n 'killed process' /var/log/messages). If so, add up all the Xmx parameters of the running java processes, add 35% and see if that total tops the total amount of memory in the machine. If it does, tweak the Xmx parameters downwards.
The easier way to test for the second is to kill the nodemanager process, keep it down and see if the problem persists (kill -9 `ps -ef | grep odeManager | awk '{print $2}'`). If the problem does not reoccur, check the WebLogic admin console on how the "Panic action" and "Failure action" are configured for each server and set them to "No Action". In that case also check the nodemanager and server logs to figure out why the node manager killed your managed server processes.
Related
I have an application which is deployed on Linux environment and has two JVM's simultaneously running. One is producer and one is consumer.
I have different targets written in my ant script for stopping and starting the two JVMs.
There are times while restarting the producer or the consumer, one of the JVMs fail to stop so we have to go and manually find the process id for that particular port and kill that process and then start the application.
How could I automate this and write one script for everything. This script should be able to call the ant targets for stopping the JVMs, kill the process if any JVMs does not stop and finally start the two JVMs.
The first and the last is fine. But how to write things like finding the process id against the port and then doing kill -9.
I am a Java developer so don't know much about this.
If your JVMs are communicating on a socket then try something like
lsof | grep ":$port " | awk '{print $2}'
where $port is the port number. This searches the list of open file descriptors for any matching the required port number and spits out the process id.
i'm with a weird problem that i tried everything and i couldn't solve it.
I have a instance of Wildfly 8.2 running a JavaEE application that controls a CallCenter, this application use like 2 ~ 8 gb memory depends on how much peopple are working, the application controls the telephony, and a web interface for configuration / reports and other sutffs.
Randomly the wildfly gets killed and i see in console the following message:
*** JBossAS process XXXX received kill signal ***
And i need to start it again.
I read about that probably being the linux OOM Killer that was killing my process, so i set in the /proc/wildfly_pid/oom_adj the value -17, as i read in documentation it makes the oom killer ignores the process, but it seems to don't work, and wildfly keeps getting killed, i did a cron job to configure the oom_adj each 1 min, and checked it, was configured correct, but nothing helps.
I was monitoring the application and the memory was like on 3 gb and its get killed, it works for some hours but randomly gets killed.
I don't know what to do, i'm using Debian 7.8 on and server that is from my client with 16gb memory and Wildfly 8.2 in standalone mode with the following java opts
-server -Xms256m -Xmx8192m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true
Any help would be very appreciated.
A link for the dmesg output dmesg
*** JBossAS process XXXX received kill signal ***
This message corresponds to Java heap dump not created on OutOfMemoryError.
This can be resolved by increasing memory limits for running your task/application.
How do we get the Java heap, memory stats and other health monitoring statistics from a Unix machine instead of from the weblogic console.
You may be able to get memory information based on the weblogic server process id. If you use
ps -ef | grep |insert weblogic server name|
to get the process ID of the weblogic server in question you can then use this document to get the memory usage of that process.
Hi I noticed that different tools for monitoring JVM require that the process will be java.exe in the taskbar. Its process id is supplied to them and then statistics regarding garbage collection and etc are returned. When running tomcat as a service, there is no a process named jave.exe but it is tomcat6.exe.
How can I get statistics regarding the JVM tomcat6 uses ?
Enable JMX in Tomcat and then connect via JConsole or JVisualVM. The name of the process is not relevant.
I'm planning to do a heap dump with jmap jdk1.5 tool on a production weblogic (10) instance.
Actually there are 3 EAR (perhaps more, don't really know i don't have access) deployed on this weblogic instance.
Someone told me "weblogic creates a JVM for each EAR"
Can someone confirm this?
With jmap i need the jvm pid as parameter to do the heap dump...
Since i have 3 EAR i guess i have 3 pid so i wonder how to know which pid correspond to which EAR JVM?
Nope - each Weblogic server (or any java process) runs in it's own JVM with it's own PID. So all your EARs will appear in the same heap dump.
If you have multiple Weblogic server instances running on the same machine, each will have a separate PID and a separate process
As #josek says, you'll have one JVM per WebLogic server, so if all your EARs are under the same WebLogic server you'l only have one pid to dump. But you may still have multiple servers - maybe an admin server and a managed server, maybe other unrelated instances - so if you just do something like ps -ef | grep java (I'm assuming this is on Unix?) you could see a lot of pids, even if you can filter it to your WebLogic's JDK_HOME.
One way to identify which pid belongs to a particular server is to go to the <domains>/servers/<your server>/tmp directory, and in there run fuser -f <your server>.lok. This will list the pids of all the processes related to that server, one of which will be the JVM java process. (May be others for JDBC etc.) One way to find just the java process (and I'm sure someone will point out another, better way!) is something like:
cd <domains>/servers/<your server>/tmp
ps -p "`fuser -f <your server>.lok 2>/dev/null`" | grep java
If each EAR is in its own server, I guess you'll have to look at config.xml to see which you need.