Log4J memory leaks, several processes taking up around a 1G each - java

Please have a look at the image below.
I am running Log4J but using the TimeAndSizeRollingFileAppender provided from http://www.simonsite.org.uk/ .
According to the site it should be a good version that is used by many companies.
What could be the cause for all these processes running independently?
I am running a WAR file, and I have this configuration:
# DEBUG < INFO < WARN < ERROR < FATAL ==> DEBUG SHOWS ALL, WARN will show WARN, ERROR and FATAL
# Variable
momomo.log.pattern=%n================================%n%d{yyyy-MM-dd-HH-mm-ss}%n%c%n%m %x%n--------------------------------%n
momomo.log.datepattern='.'yyyy-MM-dd
momomo.log.maxfilesize=5MB
# Root logger option logs all to file
log4j.rootLogger=debug, error, info, console
# global info to console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=${momomo.log.pattern}
log4j.appender.console.Threshold=INFO
# global debug to file
log4j.appender.debug=uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender
log4j.appender.debug.File=/momomo/Generated/Logs/debug.log
log4j.appender.debug.DatePattern=${momomo.log.datepattern}
log4j.appender.debug.MaxFileSize=${momomo.log.maxfilesize}
log4j.appender.debug.Append=true
log4j.appender.debug.layout=org.apache.log4j.PatternLayout
log4j.appender.debug.layout.ConversionPattern=${momomo.log.pattern}
log4j.appender.debug.Threshold=DEBUG
# global error to file
log4j.appender.error=uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender
log4j.appender.error.File=/momomo/Generated/Logs/error.log
log4j.appender.error.DatePattern=${momomo.log.datepattern}
log4j.appender.error.MaxFileSize=${momomo.log.maxfilesize}
log4j.appender.error.Append=true
log4j.appender.error.layout=org.apache.log4j.PatternLayout
log4j.appender.error.layout.ConversionPattern=${momomo.log.pattern}
log4j.appender.error.Threshold=ERROR
# This logger just logs for the package momomo.com
log4j.logger.momomo.com=momomo, momomofatal
# momomo debug logger
log4j.appender.momomo=uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender
log4j.appender.momomo.File=/momomo/Generated/Logs/momomo.log
log4j.appender.momomo.DatePattern=${momomo.log.datepattern}
log4j.appender.momomo.MaxFileSize=${momomo.log.maxfilesize}
log4j.appender.momomo.Append=true
log4j.appender.momomo.layout=org.apache.log4j.PatternLayout
log4j.appender.momomo.layout.ConversionPattern=${momomo.log.pattern}
log4j.appender.momomo.Threshold=DEBUG
# momomo fatal logger
log4j.appender.momomofatal=uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender
log4j.appender.momomofatal.File=/momomo/Generated/Logs/momomo.fatal.log
log4j.appender.momomofatal.DatePattern=${momomo.log.datepattern}
log4j.appender.momomofatal.MaxFileSize=${momomo.log.maxfilesize}
log4j.appender.momomofatal.Append=true
log4j.appender.momomofatal.layout=org.apache.log4j.PatternLayout
log4j.appender.momomofatal.layout.ConversionPattern=${momomo.log.pattern}
log4j.appender.momomofatal.Threshold=FATAL
Yes, quite many, but there is still many more processes for logging that is declared above.
The server is running really slow now, and that isn't the case when it was just started up. It has gone slower and slower for every hour.

Related

Log4j not writing logs to file on one Websphere server and writing to file on other

I have maven application with log4j.properties in it with setting to write the logs to a specified file instead of console. When I run the EAR on one of the websphere servers it is creating the file as expected and writing logs to it. But, when I am running the same EAR on other webspehere server it is writing to console instead of writing the logs to the specified file. I have checked the permissions and everything seems to be fine. Please help me in identifying what the issue is. Thanks in advance.
# CONSOLE APPENDER (stdout)
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Threshold=DEBUG
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] [%t] %-5p %20c - %m%n
# ROLLING FILE APPENDER (on the file system) for memberpolicyattributesservice code
log4j.appender.xxxxService=org.apache.log4j.RollingFileAppender
log4j.appender.xxxxService.Threshold=DEBUG
log4j.appender.xxxxService.File=/var/logs/xxxServer1/xxxServiceLog.log
log4j.appender.xxxxService.layout=org.apache.log4j.PatternLayout
log4j.appender.xxxxService.layout.ConversionPattern=%d{MM-dd#HH:mm:ss} %-5p (%13F:%L) %3x - %m%n
log4j.appender.xxxxService.MaxFileSize=10000KB
log4j.appender.xxxxService.MaxBackupIndex=30
log4j.appender.xxxxService.layout=org.apache.log4j.PatternLayout
log4j.appender.xxxxService.layout.ConversionPattern=[%d] [%t] %-5p %20c - %m%n
# ROLLING FILE APPENDER (on the file system) for hiberate, open source code log files
log4j.appender.open_source_code=org.apache.log4j.RollingFileAppender
log4j.appender.open_source_code.layout=org.apache.log4j.PatternLayout
log4j.appender.open_source_code.Threshold=DEBUG
#message format:YYYY-MM-DD HH:mm:ss,ms [ThreadId] <PRIORITY> classname.message
log4j.appender.open_source_code.layout.ConversionPattern=%d [%t]<%-5p> %c.%m \r\n
#file that will be logged to
log4j.appender.open_source_code.File=/var/logs/xxxServer1/open_source_code.log
log4j.appender.open_source_code.Append=true
log4j.appender.open_source_code.MaxFileSize=1024KB
log4j.appender.open_source_code.MaxBackupIndex=5
#turn on log4j verbose mode
log4j.debug = true
# Set root logger level to INFO and its appender to DSInstrumentor,stdout.
log4j.rootLogger=DEBUG,stdout,xxxxService
# YOUR CATEGORIES (to customize logging per class/pkg/project/etc)
log4j.category.fatal=FATAL,xxxxService
log4j.category.error=ERROR,xxxxService
#This will also enable the logging for all the children (packages and classes) of this package
log4j.logger.com.xxxxx=ALL,xxxxService
# Print only messages of level INFO in the open source code
log4j.logger.org=INFO,open_source_code
You have multiple loggers defined in your root logger (DEBUG, stdout, xxxxService) but the xxxxService loggers look like they're bound to the file system on one of the systems:
log4j.appender.open_source_code.File=/var/logs/xxxServer1/open_source_code.log
Make sure that path is valid for each server in your WAS cluster.
As a side note, you should probably avoid using debug and stdout on a remote server. This is fine for local development in your workstation, but not on a remote box. Instead, provide different log4j properties on your various deployment tiers. This lets you customize the log location or appender (say c:\temp or CONSOLE on your desktop, but /var/logs ... on all remote machines) as well as your log levels (DEBUG for desktop, maybe INFO for QA or Staging, and WARN or ERROR for Production).

log4j: logging works when package inside app, but not when package contributed by a JAR

This isn't just a rehash of how to do package-specific logging. I think I've got that nailed. It's about logging not being done when the package is moved out to a JAR shared with others.
Originally, I wrote some special logging into my web application which worked, putting special statements into a file separate from catalina.out. Later, it seemed useful to share this facility with other, cooperating web applications, so we moved the package, consisting of one interface and one class, to a utilities JAR. While the information-gathering continued successfully, the output to the "special" log file, separate from catalina.out, ceased.
When I relocate a copy of the package back into my application, the logging begins working again as originally designed.
The package name remains the same no matter whether it's physically part of my application code or linked in from a consumed library JAR.
I'm struggling to understand a class linked from a JAR is seemingly excluded from the logging scheme while it works flawlessly when simply part of the application code.
Thank you for taking the time to look at this. Any suggestions would be greatly appreciated.
Here's log4j.properties:
# --------------------------------------------------------------------------------------------
# 1) Standard Tomcat log (tomcat): /var/log/tomcat6/catalina.out
# 2) Splunkable output log (splunkable): /var/log/tomcat6/myapp.log.
# 3) Console used when debugging myapp from Eclipse.
log4j.rootLogger=TRACE,tomcat,console
# Rolling-file appender for Tomcat -----------------------------------------------------------
log4j.appender.tomcat.Threshold=INFO
log4j.appender.tomcat=org.apache.log4j.DailyRollingFileAppender
log4j.appender.tomcat.DatePattern='.'yyy-MM-dd
log4j.appender.tomcat.File=${catalina.home}/logs/catalina.out
log4j.appender.tomcat.file.MaxFileSize=500Mb
log4j.appender.tomcat.file.MaxBackupIndex=5
log4j.appender.tomcat.layout=org.apache.log4j.PatternLayout
log4j.appender.tomcat.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
# Splunkable log -----------------------------------------------------------------------------
log4j.appender.splunkable.Threshold=TRACE
log4j.appender.splunkable=org.apache.log4j.RollingFileAppender
log4j.appender.splunkable.File=${catalina.home}/logs/myapp.log
log4j.appender.splunkable.MaxFileSize=1Gb
log4j.appender.splunkable.MaxBackupIndex=7
log4j.appender.splunkable.layout=org.apache.log4j.PatternLayout
log4j.appender.splunkable.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}: %m%n
# Set the additivity flag to false to avoid propagating the rather verbose splunkable
# output to the normal Tomcat and console logs. Is this mere superstition?
log4j.category.com.acme.web.myapp.logging=TRACE,splunkable
log4j.additivity.com.acme.web.myapp.logging=false
# This logs output to the Eclipse console when running in that mode --------------------------
log4j.appender.console.Threshold=TRACE
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.SimpleLayout
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

Logging Oracle Java Stored Procedure using Java Logging API (java.util.logging )

I've developed a new Java(1.4) Stored Procedure in Oracle (10g). I've never done this before, I usually keep Java and the DB separated but this was a requisite. So I developed a Java code that can be deployed in Tomcat as well as a Stored Procedure in Oracle. For this I used the Java Logging API.
I can execute the stored procedure in Oracle, but I can't find the logs. I would like to know what should I do in order to print the logs, it can be to Oracle logs or trc files, or even, if possible, configure it to print the logs to another specific folder.
Does anyone know how to achieve this?
As appointed by PeskyGnat in this answer you have to create a logging.properties file in the folder:
$ORACLE_HOME/javavm/lib
After you configure this file, you can see the logs in:
$ORACLE_BASE/admin/$INSTANCE/udump
In my case, this was Solaris so the first path was:
/export/home/oracle/app/oracle/product/10.2.0/Db_1/javavm/lib
And the second one:
/export/home/oracle/app/oracle/admin/
As extra information, in order to make easy to find the log file you can type in Bash shell the following:
ls -altr
And this is an example of how the logging.properties file:
############################################################
# Default Logging Configuration File
#
# You can use a different file by specifying a filename
# with the java.util.logging.config.file system property.
# For example java -Djava.util.logging.config.file=myfile
############################################################
############################################################
# Global properties
############################################################
# "handlers" specifies a comma separated list of log Handler
# classes. These handlers will be installed during VM startup.
# Note that these classes must be on the system classpath.
# By default we only configure a ConsoleHandler, which will only
# show messages at the INFO and above levels.
handlers= java.util.logging.ConsoleHandler
# To also add the FileHandler, use the following line instead.
#handlers= java.util.logging.FileHandler, java.util.logging.ConsoleHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility specific level
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level= INFO
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
# default file output is in user's home directory.
java.util.logging.FileHandler.pattern = %h/java%u.log
java.util.logging.FileHandler.limit = 50000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter
# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# For example, set the com.xyz.foo logger to only log SEVERE
# messages:
**<your.class.logger>.level = <level> (i.e.: INFO)**

How to set loglevel of the Appengine DataStore in Java

I have a logging.properties file in my Java AppEngine project (using java.util.logging) that looks like this:
# Set the default logging level for all loggers to WARNING
.level = INFO
# tighten logging on the DataNucleus Categories
DataNucleus.JDO.level=WARN
# - All messages general to JDO
DataNucleus.JPA.level=WARN
# - All messages general to JPA
DataNucleus.Persistence.level=WARN
# - All messages relating to the persistence process
DataNucleus.Query.level=WARN
# - All messages relating to queries
DataNucleus.Lifecycle.level=WARN
# - All messages relating to object lifecycle changes
DataNucleus.Cache.level=WARN
# - All messages relating to the DataNucleus Cache
DataNucleus.ClassLoading.level=WARN
# - All exceptions relating to class loading issues
DataNucleus.MetaData.level=WARN
# - All messages relating to MetaData
DataNucleus.Management.level=WARN
# - All messages relating to Management
DataNucleus.General.level=WARN
# - All general operational messages
DataNucleus.Connection.level=WARN
# - All messages relating to Connections.
DataNucleus.JCA.level=WARN
# - All messages relating to Connector JCA.
DataNucleus.Transaction.level=WARN
# - All messages relating to transactions
DataNucleus.Plugin.level=WARN
# - All messages relating to DataNucleus plug-ins
DataNucleus.ValueGeneration.level=WARN
# - All messages relating to value generation
DataNucleus.Datastore.level=WARN
# - All general datastore messages
DataNucleus.Datastore.Schema.level=WARN
# - All schema related datastore log messages
DataNucleus.Datastore.Persist.level=WARN
# - All datastore persistence messages
DataNucleus.Datastore.Retrieve.level=WARN
# - All datastore retrieval messages
DataNucleus.Datastore.Native.level=WARN
# - Log of all 'native' statements sent to the datastore
DataNucleus.Enhancer.level=WARN
# - All messages from the DataNucleus Enhancer.
DataNucleus.SchemaTool.level=WARN
# - All messages from DataNucleus SchemaTool
DataNucleus.IDE.level=WARN
# - Messages from the DataNucleus IDE.
Sadly, now I am getting log messages such as:
2012-04-17 16:13:45.112
org.datanucleus.plugin.NonManagedPluginRegistry resolveConstraints: Bundle "org.datanucleus.jpa" has an optional dependency to "org.datanucleus.enhancer" but it cannot be resolved
I 2012-04-17 16:13:45.115
org.datanucleus.plugin.NonManagedPluginRegistry resolveConstraints: Bundle "org.datanucleus" has an optional dependency to "org.eclipse.equinox.registry" but it cannot be resolved
I 2012-04-17 16:13:45.719
org.datanucleus.PersistenceConfiguration setProperty: Property datanucleus.rdbms.sql.allowAllSQLStatements unknown - will be ignored
I 2012-04-17 16:13:47.806
org.datanucleus.store.appengine.MetaDataValidator validate: Performing appengine-specific metadata validation for in.animeshpathak.nazdeeq.models.Note
Can someone please tell me how I can properly configure my logging.properties file? I need the general loglevel to INFO in order to see debug messages from my code at this moment, but want to set all DataNucleus loglevels to WARN.
Thanks.
Actually, if you want to see debug messages from your code you need FINEST level, not INFO.
Try replacing your logging.properties with this:
# Set the default logging level for all loggers to FINEST
.level=FINEST
# Set the default logging level for ORM, specifically, to WARNING
DataNucleus.JDO.level=WARNING
DataNucleus.Persistence.level=WARNING
DataNucleus.Cache.level=WARNING
DataNucleus.MetaData.level=WARNING
DataNucleus.General.level=WARNING
DataNucleus.Utility.level=WARNING
DataNucleus.Transaction.level=WARNING
DataNucleus.Datastore.level=WARNING
DataNucleus.ClassLoading.level=WARNING
DataNucleus.Plugin.level=WARNING
DataNucleus.ValueGeneration.level=WARNING
DataNucleus.Enhancer.level=WARNING
DataNucleus.SchemaTool.level=WARNING
Per DataNucleus's suggestion, this is much more concise:
# Set the default logging level for all loggers to FINEST
.level=FINEST
# Set the default logging level for ORM, specifically, to WARNING
DataNucleus.level=WARNING

log4j, foreign logging

When my logger is set to "all", i am seeing messages that my code does not explicitly place. I am using a jar a friend of mine gave me to do some things (and i suspect he is logging stuff himself)
I would like to ONLY log stuff I ask to be logged by issuing the
logger.info ("something clever"); command
Below is my log4j.properties
please advise.
# ***** Set root logger level to WARN and its two appenders to stdout and R.
log4j.rootLogger=all, R
# ***** R is set to be a RollingFileAppender.
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=log/something.log
# ***** Max file size is set to 100KB
log4j.appender.R.MaxFileSize=100KB
# ***** Keep one backup file
log4j.appender.R.MaxBackupIndex=1
# ***** R uses PatternLayout.
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n
Take a look at this excellent log4j cheatsheet:
http://www.johnmunsch.com/projects/Presentations/docs/Log4J/log.properties
Basically, you need to set something like this:
log4j.rootCategory=error, R
log4j.category.com.your.package=debug
or, alternatively:
log4j.category.com.your.friends.package=error
Replace the package names as necessary.
These would make the root logger set to error (so any package - including other libraries you might be using, like Hibernate, Spring, etc.) will not log anything low-level (debug, info, warn), but log only errors.
It will also set your package (include your top level package or packages) to log on a debug level, so all your loggers will log normally. It will also set your friend's package to error, so it doesn't output anything.
If you have a common root package, just use it. E.g. if you have packages:
com.example
com.example.a
com.example.a.aa
com.example.b
com.example.c.d
just include com.example and it will inherit for the packages below by default.
You could, of course, override it, e.g. specify:
com.example - warn
com.example.a.aa - debug
or something similar. Take a look here for a detailed explanation:
http://logging.apache.org/log4j/1.2/manual.html

Categories