We Keep Coding

log4j Appender additivity for custom log level

I have a scenario where I have a separate custom level defined XYZLogLevel for logging and I have 2 rolling file appenders where 2nd appender is specifically reserved to log log messages from XYZLogLevel. However the logs are going in both the files which is undesirable.
Note:-
Logs are not package specific, so adding additivity for package wont
work.
Everything has to be done through log4j.properties file.
Adding LevelRangeFilter to first appender partially resolves it, but
when I add 3rd appender for another custom level, then I again see
duplication in 2nd and 3rd appender.
log4j.rootCategory=ERROR, F, XYZLOG, LMNLOG
log4j.appender.F=org.apache.log4j.RollingFileAppender
log4j.appender.F.File=f_log.log
log4j.appender.F.MaxFileSize=5MB
log4j.appender.F.MaxBackupIndex=10
log4j.appender.F.layout = org.apache.log4j.PatternLayout
log4j.appender.F.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss.SSS} %c{1} [%p] %m%n
log4j.appender.XYZLOG=org.apache.log4j.RollingFileAppender
log4j.appender.XYZLOG.File=xyz_reporting.log
log4j.appender.XYZLOG.threshold=XYZLOG#com.services.domain.xyzlogs.XYZLogLevel
log4j.appender.XYZLOG.filter.a=org.apache.log4j.varia.LevelMatchFilter
log4j.appender.XYZLOG.filter.a.LevelToMatch=XYZLOG#com.services.domain.xyzlogs.XYZLogLevel
log4j.appender.XYZLOG.filter.a.AcceptOnMatch=true
log4j.appender.XYZLOG.MaxFileSize=100KB
log4j.appender.XYZLOG.MaxBackupIndex=10
log4j.appender.XYZLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.XYZLOG.layout.ConversionPattern=%m%n
log4j.appender.LMNLOG=org.apache.log4j.RollingFileAppender
log4j.appender.LMNLOG.File=lmn_reporting.log
log4j.appender.LMNLOG.threshold=LMNLOG#com.services.domain.lmnlogs.LMNLogLevel
log4j.appender.LMNLOG.filter.a=org.apache.log4j.varia.LevelMatchFilter
log4j.appender.LMNLOG.filter.a.LevelToMatch=LMNLOG#com.services.domain.lmnlogs.LMNLogLevel
log4j.appender.LMNLOG.filter.a.AcceptOnMatch=true
log4j.appender.LMNLOG.MaxFileSize=100KB
log4j.appender.LMNLOG.MaxBackupIndex=10
log4j.appender.LMNLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.LMNLOG.layout.ConversionPattern=%m%n
Maybe this can be solved through org.apache.log4j.varia.DenyAllFilter but somehow I was not able to configure that for properties file. I believe that functionality is only for XML configuration.
Its a very tricky sitution as I dont have the liberty to switch to XML configuration, any help on this would be appreciated guys.
Link and examples would be great as I understand them quickly.

I am answering my own question.
First you need to create a logger and set additivity as FALSE.
log4j.logger.XZYLOG=XZYLOG#XYZLOG#com.services.domain.xyzlogs.XYZLogLevel, XYZAPPENDER
log4j.additivity.XZYLOG=false
log4j.logger.LMNLOG=LMNLOG#XYZLOG#com.services.domain.lmnlogs.LMNLogLevel, LMNAPPENDER
log4j.additivity.LMNLOG=false
Then configure the appenders in following way.
log4j.appender.XYZLOGAPPENDER=org.apache.log4j.RollingFileAppender
log4j.appender.XYZLOGAPPENDER.File=xyz_reporting.log
log4j.appender.XYZLOGAPPENDER.threshold=XYZLOG#com.services.domain.xyzlogs.XYZLogLevel
log4j.appender.XYZLOGAPPENDER.MaxFileSize=100KB
log4j.appender.XYZLOGAPPENDER.MaxBackupIndex=10
log4j.appender.XYZLOGAPPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.XYZLOGAPPENDER.layout.ConversionPattern=%m%n
log4j.appender.LMNLOGAPPENDER=org.apache.log4j.RollingFileAppender
log4j.appender.LMNLOGAPPENDER.File=lmn_reporting.log
log4j.appender.LMNLOGAPPENDER.threshold=LMNLOG#com.services.domain.lmnlogs.LMNLogLevel
log4j.appender.LMNLOGAPPENDER.MaxFileSize=100KB
log4j.appender.LMNLOGAPPENDER.MaxBackupIndex=10
log4j.appender.LMNLOGAPPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.LMNLOGAPPENDER.layout.ConversionPattern=%m%n
You may or may not need the filter in this case.
Note: I cannot post company code here thats why I modified the solution.
This is to give you general idea how I have solved this problem.

Overriding root log level java.util.logging (logging.properties)

My logging.propeties contains:
handlers = com.package.my.logging.CustomConsoleHandler
.level = INFO
org.apache.level = WARNING
org.apache.catalina.startup.DigesterFactory.level = SEVERE
Instead of setting the third-party lib levels to warning, I want to change the root log level (second line above) to .level = WARNING and set my package to com.package.my.level = INFO. However, when I do that no INFO level messages from com.package.my is ever written to the log. It seems it is not possible for my package to override the root logger level WARNING.
Is that by design? Any workaround, preferably without involving a third-party logging lib?

How to control logging by 3rd-party components in webapp in Tomcat 5.5?

We are using Tomcat 5.5 on Linux. Our webapp uses log4j for its logging (functionally-based and not many loggers), and intentionally sets the additivity of the loggers to false. Our loggers log to our own logfile. None of them log to the console.
A problem we are having is that when the level of our loggers are set to DEBUG we start getting lots of debug logging in catalina.out from 3rd party components, particularly oscache.
There is a minimal log4j.properties file in .../common/classes:
log4j.rootLogger=INFO, A1
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
# Print the date in ISO 8601 format
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
There is no log4g.properties file in .../webapps/ourapp/WEB-INF/classes.
The first thing I tried was changing to log4j.rootLogger=ERROR, A1 but that made no difference.
The next thing i tried was creating a .../webapps/ourapp/WEB-INF/classes/log4j.properties file containing the single line
log4j.logger.com.opensymphony.oscache=ERROR
to see if that would stop the oscache logging. It did, but surprisingly (to me) it stopped all the unwanted logging, not just the oscache logging. So I commented the line out and tried again. Still no unwanted logging from anything. I moved the file aside and now the unwanted logging came back. I created a zero-length log4j.properties and all the unwanted logging went away again (!) While that's short-term what I want, it makes me wonder about what other logging is being thrown away (and why!). So I'm uncomfortable just relying on that.
The "Logging" chapter in the Tomcat 5.5 docs merely says that you can do per-app configuration by putting a properties file in WEB-INF/classes but doesn't (at least not that I could find) talk about how that interacts with the configuration specified in common/classes.
So:
How are the 3rd-party components logging to catalina.out in the first place? I guess their logging could bubble up to the root logger but then why do they still log even when the root logger level is turned up to ERROR?
Why does setting the logger level on our loggers to DEBUG make this logging start? We have our own names for our loggers so there is no way the oscache and other stuff could be ancestors of our loggers.
Why does even a zero-length WEB-INF/classes/log4j.properties file stop a whole bunch of logging?
How can I do this the "right" way and limit the logging in a meaningful way rather than relying on some weird (to me) side effect to turn it off?
Curiouser and curiouser. I tried matt's suggestion of turning on debugging. I also made a more extensive log4j.properties file for the webapp:
log4j.rootLogger=INFO, SSOA1
log4j.appender.SSOA1=org.apache.log4j.ConsoleAppender
log4j.appender.SSOA1.layout=org.apache.log4j.PatternLayout
# Print the date in ISO 8601 format
log4j.appender.SSOA1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.logger.com.opensymphony.oscache=ERROR
#log4j.additivity.com.opensymphony.oscache=false
When tomcat starts up I see:
log4j: Using URL [file:/srv/www/tomcat5/base/webapps/myapp/WEB-INF/classes/log4j.properties] for automatic log4j configuration.
log4j: Reading configuration from URL file:/srv/www/tomcat5/base/webapps/myapp/WEB-INF/classes/log4j.properties
log4j: Parsing for [root] with value=[INFO, SSOA1].
log4j: Level token is [INFO].
log4j: Category root set to INFO
log4j: Parsing appender named "SSOA1".
log4j: Parsing layout options for "SSOA1".
log4j: Setting property [conversionPattern] to [%d [%t] %-5p %c - %m%n].
log4j: End of parsing for "SSOA1".
log4j: Parsed "SSOA1" options.
log4j: Parsing for [com.opensymphony.oscache] with value=[ERROR].
log4j: Level token is [ERROR].
log4j: Category com.opensymphony.oscache set to ERROR
log4j: Handling log4j.additivity.com.opensymphony.oscache=[null]
log4j: Finished configuring.
But despite the fact the oscache logger is having its level set to error, I still see stuff like this in the log:
2011-03-30 14:53:22,076 [main] DEBUG com.opensymphony.oscache.base.algorithm.AbstractConcurrentReadCache - get called (key=AUDIT_KEY_OLDEST_TIMSTAMP)
If I'm forcing the oscache logger level to ERROR (which the log4j debug output says I am), then why is this DEBUG message being sent at all? The child logger overriding the level in code?
What I did notice is that if I uncomment out the "additivity" line for the oscache logger in the webapp's log4j.properties file then the logging really does go away. So it appears the oscache logging is relying on ancestor appenders and not its own. But then it seems all the weirder that setting the oscache logger level to ERROR isn't stopping these things.

You could configure log4j.properties to get only your code logs as below and put it inot different file than catalina.out too.
handlers = org.apache.juli.FileHandler
org.apache.juli.FileHandler.level=ALL
org.apache.juli.FileHandler.directory=${catalina.base}/logs
org.apache.juli.FileHandler.prefix=yourapp-name.
com.yourproject.module.package.level=ALL

I've figured out what the heck is going on. The problem is that buried down in the bowels of another part of the webapp is the following code that is run when that component is put into debug mode:
public static synchronized void setDebugOn(boolean debugOn) {
if (isAllDebugOn() ^ debugOn) {
setAllDebugOn(debugOn);
Enumeration en = LogManager.getCurrentLoggers();
while (en.hasMoreElements()) {
setDebugOn((Logger) en.nextElement(), debugOn);
}
setDebugOn(LogManager.getRootLogger(), debugOn);
}
}
public static void setDebugOn(String name, boolean debugOn) {
setDebugOn(getLogger(name), debugOn);
}
private static void setDebugOn(Logger logger, boolean debugOn) {
logger.setLevel(debugOn ? Level.DEBUG : Level.INFO);
}
In other words, when this component is put in debug mode, it also puts EVERY SINGLE LOG4J LOGGER IN THE WEBAPP into debug mode (which I've verified by changing the code to print out the name of every logger inside of the last method of those three. So blammo -- all the 3rd-party stuff that happens to use log4j starts getting its debug output logged regardless of what log4j.properties says. Sigh.
When I change the method with that loop to only mess with the levels of specific loggers related to that component then my log4j.properties configuration starts working as expected.

Logger.setLevel() doesn't enable logging correctly

Situation: I have this log4j logger:
private static final Logger logger = Logger.getLogger(ThisClassName.class);
And am trying to set it programatically through:
Logger.getLogger(ThisClassName.class).setLevel(Level.DEBUG);
Still, DEBUG level prints are swalloed (while INFO prints are printed successfully).
Even this bit has no effect: Logger.getRootLogger().setLevel(Level.DEBUG);
Calling logger.debug("foo") reaches Category.forcedLog() and ConsoleAppender.doAppend(), and then fails (quits) at:
if(!isAsSevereAsThreshold(event.getLevel()))
Any idea why this is happening?

Your appender is configured with a threshold greater than debug, so while the logger doesn't ignore the entries, your appender doesn't record it. You need to configure the threshold of your ConsoleAppender to be DEBUG as well, either through your config file or programatically:
((ConsoleAppender)someLogger.getAppender("CONSOLE")).setThreshold(Level.DEBUG);
Config files are usually the more elegant solution for this sort of thing.
Edit: Note that apparently, any subclass of AppenderSkeleton (including ConsoleAppender) shouldn't have a threshold filter set by default. So it's likely that somewhere in your configuration you're actually manually assigning a threshold ( > Debug) to that appender, as #justkt hints.

log4j - set different loglevel for different packages/classes

I use log4j for logging and i want to print all logger.debug statements in a particular class / selected package.
i set the cfg as below>
log4j.category.my.pkg=info
log4j.category.my.pkg.ab.class1=debug
but still only info messages are shown..
is this not the right way ?

Instead of using 'category' use 'logger'. Hence, these level are configured for entire log4j, and does not depend on appender, etc.
Following change works:
log4j.logger.my.pkg=info
log4j.logger.my.pkg.ab.class1=debug

Copying from my current log4j.properties:
log4j.logger.org.hibernate.tool.hbm2ddl=warn
log4j.logger.org.hibernate.sql=info