So we ran into an interesting issue today. We have a Java EE web app that uses a bunch of 3rd party jars. This includes Hibernate.
We use the Java logging api that comes with the SDK for logging purposes. Typically we are pretty thin on logging but we ran into this issue with one of our 3rd party jars using log4j to create it's own log file. Not only was it logging statements from it's own code, it even started writing out debugs for the hibernate code which resulted in a batch job logging 3 GB worth of logs in no time.
I have 2 issues with this:
I need to fix this logging issue (If possible I do not wish to yank out or modify the log4j config in the 3rd party jar). Is there a good way to do this besides modifying or yanking out the log4j config in the 3rd party jar OR by having my own log4j config that overrides the config in the 3rd party jar? I do not like either of the 2 options but I want to do what's best.
I want to ask if it is kosher to expect 3rd party libraries to be logging away merrily? I think this is bad design. I want to get the community's opinion on this.
I want the 3rd party libraries logging ERROR's. That being said I think it is slightly awkward that this 3rd party jar to log DEBUG's from packages other than the ones it is responsible for. e.g. in my case, this 3rd party jar is logging Hibernate debugs even though it does not even call any hibernate methods. The batch job that we ran did not even call the API in this jar. It looks like having my own log4j config override what's in there seems to be the best way to go.
From what I remember you just need to provide your own log4j properties file on the classpath before your JARs that are doing the logging, and this one will be picked up. Digging back to the depths of memory, Hibernate at least picks this up from the root of its JAR, so you don't need to package your properties.
As a wider issue, yes I definitely want my libraries logging - otherwise how do you know if something has gone wrong if there's no log?
If they are using log4j or another commonly used logging framework, you can override its configuration by providing another lo4j.properties earlier on the classpath. 3rd party logging is in my opinion a good thing as long as you can easily configure the logging level that you want, which is the case if they use log4j. It's always easier to configure a logging system to output less information than to instrument 3rd party code to add logging to it !
Related
This is a follow-up (somehow of my Third-party dependencies to an OSGi application) where it was suggested that some libraries e.g. log4j are already available as bundles.
In Eclipse Indigo I could not find a log4j bundle available to Import Package as part of my installation and so I created a Plugin Project from JAR archive to bundle log4j and also a Feature Project to bundle the log4j.xml configuration following this post.
To be honest I don't understand why the fragment project is needed but this process works.
So my question now is:
Since the log4j.xml is delivered in the export as part of the feature jar, it requires some "effort" for someone to find it and update the debug levels, so I was wondering is this indeed the correct process?
I had in mind that the final exported product would deliver the log4j configuration in an easy to find location, but now (although the logging works) I am concerned whether what I do, is indeed correct.
Any help here?
If you really need to expose the file, you could put it anywhere you want, and then make sure your program calls one of these methods at startup:
org.apache.log4j.xml.DOMConfigurator.configure(String filename)
org.apache.log4j.PropertyConfigurator.configure(String
configFilename)
Or use the "configureAndWatch"-variants if you would like to make changes to the config without restarting your application.
Edit: I write "If you really need to", because I have experienced that I never need to turn on debug-logging after deployment, because it is always turned on! This is OK for applications where I have normal (but not extreme) requirements on response-time and throughput. Logging to an UDP-appender is fast (and does not fill up the disk). Or using rolling file appender is quite safe, and fast enough for my use. Always having the debug-log available is a life-saver when nailing down those hard-to-reproduce bugs.
I suggest take a look at Pax-Logging this will give you all kinds of logging frameworks for usage in a OSGi environment. And you're able to use an external configuration file (no extender needed) to configure your logging.
The fragment is one option to extend the log4j bundles classpath to include the required configuration file. It is probably the simplest way of configuring application wide properties.
This is not meant to be altered after deployment though as it will be embedded within a jar file. You will have to come up with a different approach if you expect to make it configurable after deployment.
NOTE:
I am afraid you misunderstood the answer about the jars that are already available as bundles. This does not mean that they are part of your OSGi platform of choice (Indigo), only that they are ready to be deployed to an OSGi platform as is. Your creation of a plugin project was unnecessary, you simply needed to add the jar to your target platform to resolve your missing imports.
I want to provide a client-library for wrapping rest-request to a server and log errors, so that a client can use it in his app and also see the logs. (There is also a question if i should just log the errors or rethrow it. When i use an asynchronized call (multithreaded) this might be quite tricky..)
I read that slf4j might help, because the client, who is using the library, can choose which logging framework he prefers.
Somethings puzzles me about this slf4j-thing. If he gets my library and i just provide, let's say the slf4j-api, errors will be thrown, cause the SLF4J bindings aren't included. The solution might be that he has to include the binding on his own, and the question is if he is willing to read the README for this crucial information.
If i include one "standard"-slf4j binding (e.g. the simple one), the app can't "override" this, because there is just one binding allowed on the classpath. It won't be flexible anymoe
So i am thinking to just use log4j and forget every other logging-framework. I may think to complicated on this subject, maybe someone might help me out on this?
You have to remember, your library does not set the wrapping application's classpath. The wrapping application will set a classpath that includes your library, the slf4j API library and the implementing library.
The wrapping application will take care of what slf4j implementation to use and setup all of the logging parameters. You just need to worry about logging your libraries events with the slf4j API. This is common practice, don't worry about the wrapping application.
By packaging log4j within your library, you are defeating the purpose of a logging facade. Doing so will not allow the user to pick the slf4j implementation.
I was wondering if I can create a project in eclipse or for the matter any Java IDE in which I can write my log4j initialization code and save it as a project so I can just import it in any workspace. I know how to configure a servlet in which I initialize the logger in the init() method and load the servlet on startup but that requires an entry in the web-xml which changes depending on the application.
Is there any way I can create a resuable project where there is no requirement for dependency on the DD ?
I know how to configure a servlet in which I initialize the logger in the init() method and load the servlet on startup ...
There is probably a better way.
For instance, the way I do log4j configuration on a servlet (using Tomcat) is to simply put the "log4j.properties" file on the classpath; e.g. in ".../webapps/MyApp/WEB-INF/classes/". Log4j's default strategy for locating the logging properties will find it there ... with no need for you to write any Java code.
Configuring the logging system from Java code is (IMO) a bad idea because it means that you have to change, rebuild and redeploy Java code in order to tweak the logging.
In addition to #Stephen C's answer, some application servers already come with a predefined logging congfiguration. If you're using JBoss, it already has its own log4j configuration in a file called jboss-log4j.xml, which defines a standard configuration, which you can adapt to your needs.
Other than that, I recommend to bundle a configuration with your application like described in the other answer.
Even smarter and more flexible is to use a log wrapper in your application, which will abstract from the underlying logging framework. Take a look at these:
SLF4J (preferred): http://www.slf4j.org/
Commons Logging: http://commons.apache.org/logging/
If you use one of these, you can then configure them to use the logging framework of the server you're deploying to. Many of the popular open-source frameworks use these log wrapper frameworks for similar reasons. Take a look at them, then adopt one as your standard - I strongly recommend SLF4J.
One of the major problems with using a logging framework inside your web application is that you usually end up with wanting to write to a file, and this is not allowed within the servlet API, causing you to become subtly vendor dependent and not work well with multi-computer deployments.
I would strongly suggest that you consider converting your code to use SLJF4 for your actual logging statements as it allows you to
become back end independent.
Use the "{}" placeholders to write simply log.debug("a={}, b={}", a, b) and avoid the possibly expensive generation of the actual logging string if the debug logs were not enabled without having to add a guarding if log.debugging-enabled statement.
These two along was reason enough for me to switch.
A very interesting way to handle logging then is to use the java.util.logging bridge to send all log statements to the Java log system which most web containers handle. Then the web container does all the work for you, and you can use the vendors tooling for investigating log files. Very useful!
I have a large web project that uses log4j directly, together with many 3rd-party libraries and a mix of logging libraries.
our code base - uses log4j directly.
Hibernate - uses slf4j, and the slf4j-log4j binding.
Spring - uses commons-loggings. Thus, it uses the jcl-over-slf4j bridge api, slf4j itself, and slf4j-log4j binding.
Other numerous libraries, using either commons loggings or log4j.
I am considering migrating our own code base to slf4j api, but I am not sure if the benefits are strong enough and worth the effort. Currently I am aware of the following benefits:
Cleaner api.
Performance improvements - namely the ability to use parameterized logging methods.
Ability to switch easily to logback in the future (currently logback is out of the question).
No need for additional jars, since I already have them.
Are there any other benefits? Are there any drawbacks that I am not aware of yet?
The only benefit I see for switching, is that you can funnel all the logging frameworks through only one framework, which might simplify your configuration.
Probably the main reasons why I moved to slf4j (this only applies to slf4j + logback) is that you can reload the configuration via JMX, which is GREAT when you have a problem that disappears with a server restart.
For me, there were four "killer" features which were worth the pain in moving over to Logback, on top of the ones you already mentioned (I personally switched my current major project, working flawlessly):
Automatic reloading of config files. This is awesome for production systems. If you just want to set "debug" to one class, but not bring down the whole system, this one is priceless.
Ability to use include files in the config. This allows you to have a "master" set of logging settings for all your services/JVMs, and then you can specify any packages which might require special processing. We have about ~10 services currently, and they have all a large, but mostly similar, copy of log4j.xml. We're now changing it to one master logback config file, and have that file included in every service's logback config file. The master file will still be large, but the service-specific ones should be very small. Much more maintainable.
Conditional processing. This way you can specify things like "if QAServer => then set logging levels to "DEBUG", else "INFO". Again, great for having a single config that doesn't have to change between production and dev/QA servers.
Automatic compression and deletion of old log files. You can specify how many archived files you want to keep, and optionally compress them. After creating the n+1 file, it will detect that there's one too many, and will delete the oldest one. Again, configurable per file/service, no need to do anything system-specific (batch files and/or scripts).
By the way, after doing this change, it's trivially easy to switch back to log4j. Since migrating to Logback required using SLF4J, which Log4j also supports, switching back and forth only requires you to choose which jar file to drop (Log4j's or Logback's). As long as the corresponding log4j.xml or logback config files are in the classpath, logging will work correctly.
I'm using Apache Commons Logging and SLF4J with log4j, but I also want to use the log4j.properties in a custom place like conf/log4.properties. Here is the problem:
If i use
PropertyConfigurator.configure("conf/log4j.properties");
then my app is tied to log4j and defeats the purpose of having ACL and SLF4J.
What is the best way to configure it without the app ever knowing what the logging implementation is?
I think the easiest thing to do is specify the location of the file using the log4j.configuration system property. Adopting the example in the Log4J manual:
java -Dlog4j.configuration=conf/log4j.properties -classpath ...
I believe that Log4J will find a file named "log4j.properties" anywhere on the classpath, but may be hallucinating. Worth trying, however.
As you state, by invoking PropertiesConfigurator, you are tying your application to log4j. However, the extend of this tie is quite limited. You could very easily remove the line invoking PropertiesConfigurator and recompile your code. After that is done, assuming you are using the SLF4J API for logging, you could repalce log4j with another logging framework, say logback-classic or j.u.l. just by replacing jar files. Thus, SLF4J still serves its purpose to a large extend. I would not throw the baby out with the bath water.
You can specify config file location with VM argument
-Dlog4j.configuration="file:/C:/workspace3/local/log4j.properties"