I have Log4J with SLF4J working in my app, and right now it logs the request in a standard format and it also sends default information to the syslog. I do both because there is some information that I want with each request logged - account and/or user id making the request, the request method type - post, get, etc.
Is there a way to send the same custom information to the syslog?
From what I have found you can alter the format of the log entry, but you can't append additional information to the syslog as that would screw up the format and may not be pulled into various reporting tools and you can't change the contents of a piece of the log - for example, changing the category to hold the request method type. I was thinking that route so the tools would pick up the data.
Is there a good tool that you can specify the log format and it can parse it, or it handles different log formats? I need to know more about who is doing what where so I need to create custom reports, rather than the standard - how many hits per page, etc.
Related
I have a Java application running on Google App Engine standard environment.
I am able to log from it (using JUL). In standard environment, all the application log lines from a single web request are grouped into a single entry in the request_log. Everything runs great
However, now I have a requirement to add custom labels to a log entry for a request. For example, what is user ID associated with it.
Stackdriver documentation (https://cloud.google.com/logging/docs/setup/java) gives example how to "enhance" log entries with custom labels. However, it appears that the page does not apply to standard environment.
Is it possible to add labels (or any information associated with the log entry other than app log lines) to a log entry in request_log and how? If not, what are the alternatives?
The log enhancer would allow you to add a custom labels although it would be hard coded as this function (enhanceLogEntry(LogEntry.Builder logEntry)) is called at the end of the request when the log is being populated. Supplying a value from the request to appear in the request_log made by the application would not be possible.
I do not see how it would be limited to the flexible environment, you should be able to do it from the standard environment as far as I can tell.
Alternatively, I believe the best path would be to write your own logs entries by using the Stackdriver Logging Client Libraries within your request code.
I would like to inject a piece of information into all messages logged by anything, including third-party code. The reason is that, in our web-services-oriented application, requests come in with unique IDs and I want those unique IDs to be attached to all log messages that occur while processing a request, to assist in later analysis. I am already tracking the "current request" using ThreadLocal<> techniques, so I have the ability to fetch the "current request" from anywhere.
To that end, I would like to configure log4j such that I can inject the requestID into messages before they reach the root logger (or appender?). I know that I can just make a whole new Appender that implements append() and does whatever it wants with the output, but that's not what I'm asking for. I want output to ultimately go to whatever appender is configured at startup, but with the additional information attached.
I am using log4j 1.x, but if a move to log4j 2.x or using slf4j makes this significantly easier, I would consider it.
I'm creating a Java JWS application. That application is logging some useful (for me) debug stuff to the system.out. Now, application will be used by 3rd party and i don't want them to see debug log.
My idea is following: application should write all the status messages to the custom stream. If logged user is "dev", then custom stream should be "merged" with system.out and console should print all new and prior (already existing) data from my custom stream. If logged user is somebody other than "dev", all status messages will remained logged into the custom stream and won't be visible in the console.
How could i achieve this functionality?
You can use Apache's log4j or commons logging with cutom filters. There's a Logger that comes with the JDK. You can also create a custom logger considering the types of users. You can direct the logs in proper output destination depending on the user type. All logs should be stored somewhere regardless of user type to monitor though.
I have a Swing-based application that logs all messages to text files through slf4j with logback underneath.
I'd like to add a feature to show all messages at a certain level(e.g. fatal) logged in the current session on demand, say in a JTable.
Does slf4j provide API that lets you access historical log messages, preferably filtered by level or time?
Try to use Logback, there is a ch.qos.logback.classic.db.DBAppender class that you can use as an Appender to your fatal errors. You can define your own data structure, just provide the SQL Insert statement. Also, other variants of this DBAppender are provided, so you can choose when do you want to customize the behavior.
The next thing is that you tie your appender to those loggers that you want to log.
Finally you can manage your logged data within your application (filter, purge/archive) just like your application business data.
There is only one file. And it is written simultaneously as web app copies run.
How do you filter only one session log messages from other log lines?
Using a servlet filter with either NDC or MDC information is the best way I've seen. A quick comparison of the two is available at http://wiki.apache.org/logging-log4j/NDCvsMDC.
I've found MDC has worked better for me in the past. Remember that you'll need to update your log4j properties file to include whichever version you prefer (pattern definitions at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html).
A full example of configuring MDC with a servlet filter is available at http://veerasundar.com/blog/2009/11/log4j-mdc-mapped-diagnostic-context-example-code/.
A slightly easier to configure, but significantly inferior option: You could opt to just print out the thread ID (via the properties file) for each request and make sure that the first thing you log about each request is a session identifier. It isn't as proper (or useful), but it can work for low-volume applications.
You could set a context message including the identifier of the specific app instance using org.apache.log4j.NDC, like this:
String appInstanceId = "My App Instance 1";
org.apache.log4j.NDC.push(appInstanceId);
// handle request
org.apache.log4j.NDC.clear();
You can set up the context during the initialization of your web app instance, or inside the doPost() method of your servlets. As its name implies, you can also nest contexts within contexts with multiple push calls at different levels.
See the section "Nested Diagnostic Contexts" in the Log4J manual.
Here is a page that sets up an MDC filter for web-app -> http://rtner.de/software/MDCUserServletFilter.html
Being a servlet filter it will free you from managing MDC/NDC in each of your servlets.
Of course, you should modify it to save information more pertinent to your web-app.
If you want to differentiate sessions in the same application then the MDC is the way to go. But if you want to differentiate the web applications writing to the same file, then MDC won't help because it works on a thread basis. In such case I used to make my own appender which knows which application instance it serves. This can be done through appender configuration properties. Such appender would stick application name into each logging event as a property before writing it into the media, and then you can use a layout to show this property value in the text file it writes to. Using MDC in such case won't work because every thread will have to MDC.put(applicationName) and that is quite ugly. MDC is only good for single process, not for several processes. If someone knows the other way, I'd like to hear.