I'm looking for a way to define transitive log message routing. Let's say we have an application called poly with these packages:
com.mycompany.server-common
com.mycompany.communication
com.mycompany.webservice
server-common is used by both of the 2 others. All 3 use org.hibernate as well.
Now, I like to have 1 logfile for the webservice component with all messages from com.mycompany.webservice and with those messages from com.mycompany.server-common and org.hibernate that were initiated by the webservice. And then, another coresponding file for the communication package.
My application is a war file running in tomcat, where all components run in 1 context (it comes in 1 war file). I already defined the multiple log files, but they naturally only log that what i defined statically, there is no transitive inclusion.
I would be very interested in ideas how I could achieve the desired behaviour. I already thought about using the MDC for that, but I'm not sure if that's a good idea.
Another idea was to separate the contexts, but I think in the current project state this will be hard and it does not offer the flexibility I hope for.
Any hints or discussions are appreciated.
If you set an MDC key when webservice starts serving a request and clear the MDC key at the end of the request, SiftingAppender will do what you are asking. Shout on the logback-user mailing list if you run into difficulties.
Related
We are using cas-client-core-3.3.3.jar for providing single sign on functionality in our application and we are trying to emit this jar library logs into our appliation logs.
Our application is a weblogic based application and we are using log4j for logging.
So to get cas-client-core-3.3.3.jar logs in our application log we have added this property in our log4j.properties
log4j.logger.org.jasig.cas=DEBUG
but we are not getting the logs which are expected from org.jasig.cas classes. I am attaching sample log here which is expected
2015-05-13 10:00:17,798 DEBUG [org.jasig.cas.client.validation.Saml11TicketValidator.<constructValidationUrl>] - Placing URL parameters in map.
2015-05-13 10:00:17,801 DEBUG [org.jasig.cas.client.validation.Saml11TicketValidator.<constructValidationUrl>] - Calling template URL attribute map.
2015-05-13 10:00:17,802 DEBUG [org.jasig.cas.client.validation.Saml11TicketValidator.<constructValidationUrl>] - Loading custom parameters from configuration.
2015-05-13 10:00:17,803 DEBUG [org.jasig.cas.client.validation.Saml11TicketValidator.<validate>] - Constructing validation url:
Disclaimer: I've never worked with Web Sphere but worked a lot with different logging systems, so my answer is based on my experience in this area.
First off, cas uses slf4j under the hood which is great.
Slf4j is only an interface (slf4j-api jar), and if you want to use it with log4j which is a concrete implementation of logging system that knows nothing about slf4j apis you should provide an implementation of sfl4j interfaces that will delegate the calls to log4j loggers.
So you should also include such an adapter in classpath as well Here is the link.
Now if this doesn't work, then probably the log4j.properties are not configured correctly, for example, the logger doesn't have any associated appenders/wrong appenders.
I've found the best way to check this is just to place a breakpoint on the logger's call (inside cas library) and see the following:
Which implementation of slf4j interface is actually used (as I've said before org.sl4j.Logger is just an interface and it has to be instantiated with real implementation object somehow, you know)
See the associated appenders to the underlying implementation.
Regarding the second item, depending on technology/frameworks you have, you might be able to get this information via JMX or some kind of web admin interface. Debugging is a "hardcore" general way to figure this out.
I need to split 1 batch application to 3 different. Code is almost the same, I have just modified ANT build script, and excluded or included some dependencies for different app. Than I have set different web.xml for each war. Each web.xml defines different spring application context with different beans for different behaviour.
All wars run on one tomcat server. Application used log4j, but now I refactored it to use slf4j instead. Thought I still need to use log4j under slf4j.
The problem I have is that each application log must appear in different log file,
even though class names are the same.
I can't write different log4j.properties file, because administrators placed it in tomcat/lib folder for all applications.
I have tried to place 3 files in tomcat/lib and change configuration file name for each application when initializing servlet, but it changed for all applications at same time.
Only solution I can think of now is to wrap log4j-over-slf4j, create 3 different slf4j log factories, that would append some prefix for each log name. For example, if I have this log:
private final Logger logger = LoggerFactory.getLogger(MainProcessor.class);
Each logging factory would genarate these logging names (with prefixes app1,app2 and app3) :
app1.com.test.MainProcessor
app2.com.test.MainProcessor
app3.com.test.MainProcessor
Is there any better way to deal with this problem ?
Try using a hook method, fire event, etc. so the logging doesn't happen in the class that is shared across the applications, but in some (top) class that is unique per application.
Variation is to statically access some logging class, use a singleton, etc. from the class where the logging should occur, but set context to that logging class on app initialization.
Executive Summary
How do I filter by the servlet in which the log message was invoked? (presently using 2.0 beta8)
Why on earth I would want to do that...
I have several existing web applications. They were written to rely on a proprietary logging system. I have re-implemented a key class from the proprietary system from scratch and added it as a class the proprietary system as a jar and log4j 2.0 as jars in tomcat, thereby utilizing the class loading load order in tomcat to divert the proprietary system into log4j. This succeeds and my log4j config now controls everything (Yay!).
But... (There's always a "But"!)
I was very pleased until I discovered that with all 4 applications deployed in the same container, they were not coordinating their writes to the single log file in the single configuration I had placed in conf/log4j2.xml (and specifed by passing -Dlog4j.configurationFile=/mnt/kui/tomcat/conf/log4j2.xml on the command line). I found some log messages with much earlier time stamps (hours earlier) in the middle of the log file. Out of order logs (and possibly overwritten log lines?) are not desirable of course.
I actually don't want them all in one file anyway and would prefer a log per application controlled by a single config file. Initially I thought this would be easy to achieve since log4j automatically sets up a LoggingContext with the name of the web application.
However I can't seem to find a filter implementation that will allow me to filter on the LoggingContext. I understand that from each application's perspective there is only one logging context (I think), but the same config file is read by 4 applications so from the config perspective LoggingContext is not unique.
I'm looking for a way to route each application to it's own file without having a config file for every application, or having to add classes to all the applications or edit war files (including web.xml). I'm sooo... close but It's not working.
Just to complicate matters, there is a jar file we wrote that is shared among all 4 applications that uses this logging too and one application has converted to using log4j directly in it's classes (but it still uses proprietary classes that reference the proprietary logging class that I replaced).
I have already seen http://logging.apache.org/log4j/2.x/manual/logsep.html and my case seems closest to '"Shared" Web Applications and REST Service Containers' but that case doesn't seem very well covered by that page.
You may want to look at the RoutingAppender which can be used to separate log files based on data in your ThreadContextMap. You could use the web app name as a unique key.
About the out of order logs, there was an issue with FastFileAppender in older betas. If append was false, the old file was not truncated but new log events would start to overwrite the old file from the beginning. (So after your most recent log event you would see yesterday's log events, for example). What version are you using?
There is only one file. And it is written simultaneously as web app copies run.
How do you filter only one session log messages from other log lines?
Using a servlet filter with either NDC or MDC information is the best way I've seen. A quick comparison of the two is available at http://wiki.apache.org/logging-log4j/NDCvsMDC.
I've found MDC has worked better for me in the past. Remember that you'll need to update your log4j properties file to include whichever version you prefer (pattern definitions at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html).
A full example of configuring MDC with a servlet filter is available at http://veerasundar.com/blog/2009/11/log4j-mdc-mapped-diagnostic-context-example-code/.
A slightly easier to configure, but significantly inferior option: You could opt to just print out the thread ID (via the properties file) for each request and make sure that the first thing you log about each request is a session identifier. It isn't as proper (or useful), but it can work for low-volume applications.
You could set a context message including the identifier of the specific app instance using org.apache.log4j.NDC, like this:
String appInstanceId = "My App Instance 1";
org.apache.log4j.NDC.push(appInstanceId);
// handle request
org.apache.log4j.NDC.clear();
You can set up the context during the initialization of your web app instance, or inside the doPost() method of your servlets. As its name implies, you can also nest contexts within contexts with multiple push calls at different levels.
See the section "Nested Diagnostic Contexts" in the Log4J manual.
Here is a page that sets up an MDC filter for web-app -> http://rtner.de/software/MDCUserServletFilter.html
Being a servlet filter it will free you from managing MDC/NDC in each of your servlets.
Of course, you should modify it to save information more pertinent to your web-app.
If you want to differentiate sessions in the same application then the MDC is the way to go. But if you want to differentiate the web applications writing to the same file, then MDC won't help because it works on a thread basis. In such case I used to make my own appender which knows which application instance it serves. This can be done through appender configuration properties. Such appender would stick application name into each logging event as a property before writing it into the media, and then you can use a layout to show this property value in the text file it writes to. Using MDC in such case won't work because every thread will have to MDC.put(applicationName) and that is quite ugly. MDC is only good for single process, not for several processes. If someone knows the other way, I'd like to hear.
So, I'm beating my head against the wall with logging again. I know, how complex can it be? Well, let's see...
I'm starting a new project to be run on WebSphere Application Server 6.1 (actually Portal Server 6.1, but it's WAS 6.1 under the hood - whatever). I usually use java.util.logging for my WAS projects and everything is fine. This customer is a SLF4J fan and wants to use that. Fair enough, sounds easy.
So, I deploy slf4j-api-1.5.8.jar and slf4j-jdk14-1.5.8.jar in my WEB-INF/lib directory. In my code I do a --
// These classes are coming from org.slf4j.*
private static Logger log = LoggerFactory.getLogger(MyClass.class);
...
log.debug("This is a log message");
As expected I get an entry in the SystemOut.log. However, it's the format of that message that I can't figure out. A sample would be --
[12/15/09 15:43:15:071 EST] 00000042 MyClass D com.example.MyClass This is a log message
Let me explain what's in that sample log entry. I assume everything to the left of com.example.MyClass is coming from the j.u.l formatter. Everything to the right of it is what I included in my log.debug(). So, who's adding the com.example.MyClass? Only thing I can think is that SLF4J is adding it before it passes the message through to the underlying j.u.l.
It's the com.example.MyClass part that's irritating me. I don't want that included in the SLF4J-generated log message. The class name is already included, so it's extra fluff that's not needed. Plus, the real package names are quite long and their inclusion just pushes the real meat of the log entry too far off to the right.
When I use just plain java.util.loggging, the log entry is exactly the same except that the "com.example.MyClass" piece is not included. Exactly as I want!
So, the question is - how can I get rid of this extra class name entry in the log messages generated via SLF4J under WAS?
Thanks!
You bind slf4j to java.util.logging which most likely is configured inside WebSphere as it doesn't look like the standard message format.
I do not know WebSphere, but you may get a better result by telling slf4j to bind to something else. Would the slf4j-simple backend do? It just prints out info-or-higher messages instead of invoking java.util.logging.
Basically you want to configure the layout of the logging messages produced by the underlying logging mechanism.
SLF4J does not actually perform the logging, but delegates to other logging systems (log4j, JUL, etc) based on how you set it up.
So if you are binding SLF4J with JUL, then I think the real question you are asking is either one of
Setting the Formatter of a Logger Handler
Creating a Custom Formatter for a Logger Handler