I have a large web project that uses log4j directly, together with many 3rd-party libraries and a mix of logging libraries.
our code base - uses log4j directly.
Hibernate - uses slf4j, and the slf4j-log4j binding.
Spring - uses commons-loggings. Thus, it uses the jcl-over-slf4j bridge api, slf4j itself, and slf4j-log4j binding.
Other numerous libraries, using either commons loggings or log4j.
I am considering migrating our own code base to slf4j api, but I am not sure if the benefits are strong enough and worth the effort. Currently I am aware of the following benefits:
Cleaner api.
Performance improvements - namely the ability to use parameterized logging methods.
Ability to switch easily to logback in the future (currently logback is out of the question).
No need for additional jars, since I already have them.
Are there any other benefits? Are there any drawbacks that I am not aware of yet?
The only benefit I see for switching, is that you can funnel all the logging frameworks through only one framework, which might simplify your configuration.
Probably the main reasons why I moved to slf4j (this only applies to slf4j + logback) is that you can reload the configuration via JMX, which is GREAT when you have a problem that disappears with a server restart.
For me, there were four "killer" features which were worth the pain in moving over to Logback, on top of the ones you already mentioned (I personally switched my current major project, working flawlessly):
Automatic reloading of config files. This is awesome for production systems. If you just want to set "debug" to one class, but not bring down the whole system, this one is priceless.
Ability to use include files in the config. This allows you to have a "master" set of logging settings for all your services/JVMs, and then you can specify any packages which might require special processing. We have about ~10 services currently, and they have all a large, but mostly similar, copy of log4j.xml. We're now changing it to one master logback config file, and have that file included in every service's logback config file. The master file will still be large, but the service-specific ones should be very small. Much more maintainable.
Conditional processing. This way you can specify things like "if QAServer => then set logging levels to "DEBUG", else "INFO". Again, great for having a single config that doesn't have to change between production and dev/QA servers.
Automatic compression and deletion of old log files. You can specify how many archived files you want to keep, and optionally compress them. After creating the n+1 file, it will detect that there's one too many, and will delete the oldest one. Again, configurable per file/service, no need to do anything system-specific (batch files and/or scripts).
By the way, after doing this change, it's trivially easy to switch back to log4j. Since migrating to Logback required using SLF4J, which Log4j also supports, switching back and forth only requires you to choose which jar file to drop (Log4j's or Logback's). As long as the corresponding log4j.xml or logback config files are in the classpath, logging will work correctly.
Related
Jetty 9 is used for the embedded server and everything works well. One thing that remains is the logging issue.
Prior to that mvn:jetty-run brings his own logging setup with it and logs to the console. That is good for development. In the production environment we need something more special.
Currently on start-up the SLF4J complains about, that there is no binding available, so we can chose freely.
That is what we want to archive:
We need to log to the console if we are starting in a non-production environment.
In the production environment the logging should be done in a single log-file but on a daily rotation with the naming schema: logs/logname-date.log (e.g. logs/application-20130926.log)
We distinguish between the production and non-production mode using a command line argument '-production'.
Since the jetty server is embedded I would love to have a solution which we can fully configure the logger without the need to manage xml or properties-files taking the logging configuration aspect out of the deployment process.
So what options do we have and how can we do this in the best possible way?
Update: It seems that logback is the way to go. It has support for the logfile rotation and also makes it possible to use a console output. The difficult question remaining is how to do this programatically and without additional files.
You have hundreds of configuration options here.
You will need to know a few things about your application before you can pick an appropriate configuration here.
How are logging events emitted from your code? the jetty server? and all 3rd party libraries?
Then you want to answer, what logging framework do you want to handle output (to disk, and to console) portions of the logging architecture?
This is documented at Jetty:
http://www.eclipse.org/jetty/documentation/current/example-logging-logback-centralized.html
Yes, that documentation isn't for embedded mode, but it is still relevant.
Your required logging jar files:
The basic api jar:
slf4j-api.jar (required, no matter what you choose below)
The log capturing jars:
(pick [0..n] jars here)
log4j-over-slf4j.jar (slf4j handling of log4j events)
jul-to-slf4j.jar (slf4j handling of java.util.logging events)
jcl-over-slf4j.jar (slf4j handling of jakarta commons-logging events)
The log output jars:
Pick only 1 of the following output jars:
slf4j-simple.jar (this is a super simple logging implementation, not suitable for production)
logback-classic.jar (slf4j's own output logging framework)
also requires logback-core.jar
slf4j-log4j12.jar (route slf4j events to log4j for processing)
also requires log4j.jar
do not use if using log4j-over-slf4j.jar from above
slf4j-jdk14.jar (route slf4j events to java.util.logging for processing)
do not use if using jul-to-slf4j.jar from above
slf4j-nop.jar (route slf4j events to nowhere, silently discard them)
slf4j-jcl.jar (route slf4j events to jakarta commons-logging)
also requires commons-logging.jar and your choice of commons-logging implementation.
do not use if using jcl-over-slf4j.jar from above
Configure it all:
Be sure you read the slf4j manual on each of these jars, as there is sometimes some extra setup details you might need to know about.
For your described situation, the most appropriate output jars would be logback-classic.jar or slf4j-log4j12.jar. As for configuring the output, you would need to rely on the documentation that those libraries provide.
Logback Documentation
Log4j Wiki
So finally here is the complete picture.
After all the logging configuration in a programmatic way is just described here: http://logback.qos.ch/manual/configuration.html#joranDirectly
I use the logback API just as stated by Joakim. Once you learn how to program it programmatically using the JoranConfigurator object everything is quite easy. Play with it and you get the picture.
I managed to accomplished all tasks at hand.
Thanks for the help Joakim. I was missing the JoranConfigurator thingy. Thanks!
Update:
I used a StringReader and embedded the xml configuration file directly in the Logging configuration class. This way I dont have to manage additional files and logging works as expected.
I was wondering if I can create a project in eclipse or for the matter any Java IDE in which I can write my log4j initialization code and save it as a project so I can just import it in any workspace. I know how to configure a servlet in which I initialize the logger in the init() method and load the servlet on startup but that requires an entry in the web-xml which changes depending on the application.
Is there any way I can create a resuable project where there is no requirement for dependency on the DD ?
I know how to configure a servlet in which I initialize the logger in the init() method and load the servlet on startup ...
There is probably a better way.
For instance, the way I do log4j configuration on a servlet (using Tomcat) is to simply put the "log4j.properties" file on the classpath; e.g. in ".../webapps/MyApp/WEB-INF/classes/". Log4j's default strategy for locating the logging properties will find it there ... with no need for you to write any Java code.
Configuring the logging system from Java code is (IMO) a bad idea because it means that you have to change, rebuild and redeploy Java code in order to tweak the logging.
In addition to #Stephen C's answer, some application servers already come with a predefined logging congfiguration. If you're using JBoss, it already has its own log4j configuration in a file called jboss-log4j.xml, which defines a standard configuration, which you can adapt to your needs.
Other than that, I recommend to bundle a configuration with your application like described in the other answer.
Even smarter and more flexible is to use a log wrapper in your application, which will abstract from the underlying logging framework. Take a look at these:
SLF4J (preferred): http://www.slf4j.org/
Commons Logging: http://commons.apache.org/logging/
If you use one of these, you can then configure them to use the logging framework of the server you're deploying to. Many of the popular open-source frameworks use these log wrapper frameworks for similar reasons. Take a look at them, then adopt one as your standard - I strongly recommend SLF4J.
One of the major problems with using a logging framework inside your web application is that you usually end up with wanting to write to a file, and this is not allowed within the servlet API, causing you to become subtly vendor dependent and not work well with multi-computer deployments.
I would strongly suggest that you consider converting your code to use SLJF4 for your actual logging statements as it allows you to
become back end independent.
Use the "{}" placeholders to write simply log.debug("a={}, b={}", a, b) and avoid the possibly expensive generation of the actual logging string if the debug logs were not enabled without having to add a guarding if log.debugging-enabled statement.
These two along was reason enough for me to switch.
A very interesting way to handle logging then is to use the java.util.logging bridge to send all log statements to the Java log system which most web containers handle. Then the web container does all the work for you, and you can use the vendors tooling for investigating log files. Very useful!
So we ran into an interesting issue today. We have a Java EE web app that uses a bunch of 3rd party jars. This includes Hibernate.
We use the Java logging api that comes with the SDK for logging purposes. Typically we are pretty thin on logging but we ran into this issue with one of our 3rd party jars using log4j to create it's own log file. Not only was it logging statements from it's own code, it even started writing out debugs for the hibernate code which resulted in a batch job logging 3 GB worth of logs in no time.
I have 2 issues with this:
I need to fix this logging issue (If possible I do not wish to yank out or modify the log4j config in the 3rd party jar). Is there a good way to do this besides modifying or yanking out the log4j config in the 3rd party jar OR by having my own log4j config that overrides the config in the 3rd party jar? I do not like either of the 2 options but I want to do what's best.
I want to ask if it is kosher to expect 3rd party libraries to be logging away merrily? I think this is bad design. I want to get the community's opinion on this.
I want the 3rd party libraries logging ERROR's. That being said I think it is slightly awkward that this 3rd party jar to log DEBUG's from packages other than the ones it is responsible for. e.g. in my case, this 3rd party jar is logging Hibernate debugs even though it does not even call any hibernate methods. The batch job that we ran did not even call the API in this jar. It looks like having my own log4j config override what's in there seems to be the best way to go.
From what I remember you just need to provide your own log4j properties file on the classpath before your JARs that are doing the logging, and this one will be picked up. Digging back to the depths of memory, Hibernate at least picks this up from the root of its JAR, so you don't need to package your properties.
As a wider issue, yes I definitely want my libraries logging - otherwise how do you know if something has gone wrong if there's no log?
If they are using log4j or another commonly used logging framework, you can override its configuration by providing another lo4j.properties earlier on the classpath. 3rd party logging is in my opinion a good thing as long as you can easily configure the logging level that you want, which is the case if they use log4j. It's always easier to configure a logging system to output less information than to instrument 3rd party code to add logging to it !
I have some questions about logging, more specifically about setting it up and making sure it works.
The project I'm doing will use Wicket, Spring and Hibernate. I know that Wicket and Hibernate uses Simple Logging Facade for Java (SL4J) and that Spring is using the logging component from Apache Commons.
Will they co-exist happily?
I thought I would use log4j together with both SL4J and the logging component from Apache commons, do you think that's a good idea?
Can I set up them all to output logging data into a common file?
Or should I use separate files?
Or should I store the logging messages in the database? (I'd rather not, as I find grepping etc on text files quite convenient.)
For Spring I guess I need some kind of configuration file for the Apache Commons logging component as well where I direct it to use log4j?
When I've set these up I guess to see that everything works I set the logging level to INFO as it's fairly certain that all three of the frameworks output some information in that mode? Or is there an even better way to make sure?
And my last question. In the project I'm starting, do you recommend that I use SL4J for my own logging purposes? (I thought I would use log4j directly, but that was before I learned a little bit more about logging and a lot of respectable libraries seem to choose the path of a bridge/facade for their logging needs. And if it gets us flexibility without added cost there's no reason not to do it that way.)
I'm looking forward to hearing more from you about how you are doing your logging. It's a new area for me which I'm eager to improve myself in.
Well SLF4J is just a facade, like commons logging, which means they still need something else to work. They permit library authors to not force users into having multiple logging library and configuration. Log4j and logback are regular logging libs.
See here for more info.
SLF4J has a commons logging bridge that you can use to replace the commons logging library. I think the schema there explain the situation very well.
Now, you just need to use slf4j-logj12.jar to have commons logging and slf4j use log4j (or anything else you chose; btw, logback doesn't need an additional library to be used with slf4j) as a backing engine.
You application will thus have
jcl104-over-slf4j.jar (to bridge jakarta commons logging to slf4j)
slf4j.jar (for hibernate and others to use slf4j)
slf4j-logj12.jar (for slf4j to use log4j as a backend)
log4j.jar (for your application to use. all config will also be done here)
Here is how to redirect everything to SLF4J:
remove commons-logging.jar from your classpath. If you are using Maven and have trouble getting rid of commons-logging, see this.
put jcl-over-slf4j.jar in your classpath (it comes in the SLF4J distribution). This is a drop-in replacement that mimics JCL's classes, but calls SLF4J internally. This will take care of Spring, and any other framework that uses JCL.
Connect SLF4J to your favorite backend (Log4J, Logback...) by putting slf4j-xxx.jar in the classpath. Configure the backend to log all categories to one file, and you're done.
As for using SLF4J in your application, it is not strictly necessary. Libraries like JCL and SLF4J were originally designed for people who write libraries and do no want to lock their clients into a particular logging framework.
PS: by the way, JCL = Jakarta Commons Logging
Should new projects use logback instead of log4j as a logging framework ?
Or with other words :'Is logback better than log4j (leaving the SLF4J-'feature' of logback beside)?'
You should use SLF4J+Logback for logging.
It provides neat features like parametrized messages and (in contrast to commons-logging) a Mapped Diagnostic Context (MDC, javadoc, documentation).
Using SLF4J makes the logging backend exchangeable in a quite elegant way.
Additionally, SLF4J supports bridging of other logging frameworks to the actual SLF4J implementation you'll be using so logging events from third party software will show up in your unified logs - with the exception of java.util.logging that can't be bridged the same way that other logging frameworks are.
Bridging jul is explained in the javadocs of SLF4JBridgeHandler.
I've had a very good experience using the SLF4J+Logback combination in several projects and LOG4J development has pretty much stalled.
SLF4J has the following remaining downsides:
It does not support varargs to stay compatible with Java < 1.5
It does not support using both parametrized message and an exception at the same time.
It does not contain support for a Nested Diagnostic Context (NDC, javadoc) which LOG4J has.
The author (of both Logback and Log4j) has a list of reasons to change at http://logback.qos.ch/reasonsToSwitch.html.
Here are a few that stuck out at me;
Faster implementation
Based on our previous work on log4j,
logback internals have been
re-written to perform about ten times
faster on certain critical execution
paths. Not only are logback
components faster, they have a
smaller memory footprint as well.
Automatic reloading of configuration
files
Logback-classic can automatically
reload its configuration file upon
modification. The scanning process is
both fast and safe as it does not
involve the creation of a separate
thread for scanning. This technical
subtlety ensures that logback plays
well within application servers and
more generally within the JEE
environment.
Stack traces with packaging data
When logback prints an exception, the
stack trace will include packaging
data. Here is a sample stack trace
generated by the logback-demo
web-application.
14:28:48.835 [btpool0-7] INFO
c.q.l.demo.prime.PrimeAction - 99 is
not a valid value
java.lang.Exception: 99 is invalid
at
ch.qos.logback.demo.prime.PrimeAction.execute(PrimeAction.java:28)
[classes/:na] at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
[struts-1.2.9.jar:1.2.9] at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
[struts-1.2.9.jar:1.2.9] at
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
[struts-1.2.9.jar:1.2.9] at
javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
[servlet-api-2.5-6.1.12.jar:6.1.12]
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
[jetty-6.1.12.jar:6.1.12] at
ch.qos.logback.demo.UserServletFilter.doFilter(UserServletFilter.java:44)
[classes/:na] at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
[jetty-6.1.12.jar:6.1.12] at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
[jetty-6.1.12.jar:6.1.12] at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
[jetty-6.1.12.jar:6.1.12] at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
[jetty-6.1.12.jar:6.1.12]
From the above, you can recognize
that the application is using Struts
version 1.2.9 and was deployed under
jetty version 6.1.12. Thus, stack
traces will quickly inform the reader
about the classes invervening in the
exception but also the package and
package versions they belong to. When
your customers send you a stack
trace, as a developer you will no
longer need to ask them to send you
information about the versions of
packages they are using. The
information will be part of the stack
trace. See "%xThrowable" conversion
word for details.
This feature can be quite helpful to
the point that some users mistakenly
consider it a feature of their IDE.
Automatic removal of old log archives
By setting the maxHistory property of
TimeBasedRollingPolicy or
SizeAndTimeBasedFNATP, you can
control the maximum number of
archived files. If your rolling
policy calls for monthly rollover and
you wish to keep one year's worth of
logs, simply set the maxHistory
property to 12. Archived log files
older than 12 months will be
automatically removed.
There may be a bias there, but the same guy did write both frameworks and if he is saying use Logback over Log4j he's probably worth listening to.
I would use slf4j for logging in all cases. This allow you to choose which actual logging backend you want to use, at deploy time instead of code time.
This has proven to be very valuable to me. It allows me to use log4j in old JVM's, and logback in 1.5+ JVM's, and also java.util.logging if needed.
Logback more Java EE aware:
in general (from code to documentation) it’s keeping in mind containers – how multiple apps coexist, how class loaders implemented etc. Contexts for loggers, JNDI, JMX configuration included etc.
from developer prospective almost same, Logback adds
Parameterized logging (no need to use if(logger.isDebugEnabled()) to avoid string concatenation overhead )
Log4j – only giant plus is old JVM support, small (IMO)
NDC (Logback only MDC), some extensions. For example I wrote extension for configureAndWatch for Log4j, no such thing for Logback
the original log4j and logback were designed and implemented by the same guy.
several open source tools have used SLF4J. I don't see any significant deficiencies in this tool. So unless you have a lot extensions to log4j in your codebase, I would go ahead with logback.
I would think that your decision should come down to the same one it would if you were deciding between using log4j or Jakarta Commons Logging - are you developing a library which will be included in other applications? If so, then it doesn't seem fair to force users of your library to also use your logging library of choice.
If the answer is no, I would just go with what is simpler to add and what you are more comfortable with. Sounds like logback is just as extensible and reliable as log4j, so if you're comfortable using it, go ahead.
I'm not familiar with SLF4J, and I've only taken a brief look at logback, but two things come to mind.
First, why are you excluding a tool from examination? I think it's important to keep an open mind and examine all possibilities to choose the best one.
Second, I think that in some projects one tool is a better than another tool, and the opposite might be true in a different project. I don't think that one tool is always better than another tool. There is, after all, no silver bullet.
To answer your question - Yes and no. It depends on the project, and how familiar the team is with one tool. I wouldn't say "don't use log4j" if the entire team is very comfortable with it, it meets all the needs, and logback doesn't offer anything that we need to complete the task.