We understand that Jetty 11 has basically changed logging from version 10 (no internal Jetty classes, moreover Jetty 11 is commited to use SLF4 as a base logging).
The problem
We have a rudimentary knowledge of SLF4J (used it before and we've even read the Jetty 11 SLF4J sources ,too), but currently we don't see any way to "teach" Jetty 11 a new logging (aka there are no "setLogging()" methods in the Jetty 11 source code as there were before).
Global (Jetty) parameters, alas, can't be our solution just yet.
The state (aka our requirements)
We have already solved the "RequestLog" outputs of Jetty, no problems there, we need the "normal" Jetty-log outputs.
We need to control (many) modules/jars etc. via a unified logging.
Our logging is simple but requires that no output happens on the console (stdout / stderr etc). In best case the logging gets an instance of an Exception/Runtime, too.
Therefore, we need to route the Jetty output from the "Jetty server" through our internal logging. Using SLF4? If there is no other way (and we see no other way up to now), gladly.
Switching back to Jetty 10, sadly, is not an option.
Could this be solved in any way we are not aware of (yet)? Any idea would be very appreciated, thank you!
The switch from Jetty logging to Slf4j was actually done in Jetty 10.0.0.
slf4j was designed for unified logging, it can capture into a single logging location implementation all of the logging events generated from libraries that use ...
slf4j API
java.util.logging API
log4j1 API
log4j2 API
commons-logging API
logback API
org.apache.juli.logging API
and if you use slf4j version 2.x series, there's even rudimentary support for capturing java.lang.System.Logger API.
With slf4j, you have 2 categories of jar files to think about.
Bridge API JARs
These are slf4j based JARs that merely capture the above logging events and route them to slf4j. You can choose 0..n of these JARs to use.
There's dozens of options here.
Here's some common ones
jcl-overs-slf4j - captures Jakarta Commons Logging events and sends to slf4j
jul-to-slf4j - captures Java Util Logging events and sends them to slf4j
log4j-over-slf4j - captures Log4j 1.x events and sends them to slf4j
log4j2-overs-slf4j - captures Log4j 2.x events and sends them to slf4j
osgi-over-slf4j - captures osgi logging bundle events and sends them to slf4j
See http://www.slf4j.org/legacy.html
Implementation Binding JAR
These are the implementation of slf4j-api, and is the final binding of all logging events, it is the thing that decides what to do with the logging event (eg: write it to disk, ignore it, send it to a logging database, etc)
You have many choices here as well, here's some common jars to pick from (pick only 1!)
logback-classic - slf4j to Logback (Eclipse Jetty group's favorite logging implementation)
slf4j-jdk14 - slf4j to Java Util Logging
slf4j-log4j12 - slf4j to Log4j 1.2.x
log4j-slf4j-impl - slf4j to Log4j 2.x (see https://logging.apache.org/log4j/2.x/log4j-slf4j-impl/)
slfj-jcl - slf4j to Jakarta Commons Logging
jetty-slf4j-impl - Jetty 10+ implementation of the slf4j api
See: http://www.slf4j.org/manual.html#swapping
Since Jetty 10.0.x, the jetty-slf4j-impl exists, which provides an out of the box implementation that simply writes to System.err (aka STDERR) with some decent logging filtering by level in the usual jetty-logging.properties.
See https://search.maven.org/artifact/org.eclipse.jetty/jetty-slf4j-impl
Important advice
Don't use multiple binding implementations. Narrow it down to 1 binding implementation and purge all other logging implementation jars.
Don't accidentally create a loop with introducing a Bridge API Jar and a Binding Implementation JAR with the same logging technology. (eg: using binding log4j-over-slf4j and slf4j-log4j12 at the same time)
There is no "configuration" to wire up these binding or bridge jars, their mere existence in the classloader is enough to make them work. See the slf4j manual on how that works.
We have already solved the "RequestLog" outputs of Jetty, no problems there, we need the "normal" Jetty-log outputs.
Interesting, this is "solved" by actually using slf4j, as that's the only non-deprecated implementation of RequestLog.Writer in Jetty 10 and Jetty 11.
The way this works, is the Slf4jRequestLogWriter will emit events to a single named logger (the name of which you can configure in Slf4jRequestLogWriter.setLoggerName(String)) using the slf4j-api. Then it reaches the logging implementation and is routed wherever that logging configuration decides based on that logger name (file, with rolling, syslog, sent to a different system for aggregation, logstash, etc)
Did you really implement your own RequestLog.Writer instead of just using your preferred logging logging library? (libraries like logback, log4j2, log4j1, and even java.util.logging can easily create separate log files for RequestLog events).
⚠️ Note: do not use logback-access for RequestLog at this time (It does not fully support jakarta.servlets yet, and has many bugs that result in bad request log data. See open PR at https://github.com/qos-ch/logback/pull/532)
What is the fastest async way to log to file in Vert.x?
The aim is to write logs from loggers from different classes (i.e. Class1, Class2 etc) to 1 file (something like 'console.log')
Vert.x uses the JDK bundled JUL logging framework to avoid shipping additional dependencies. However it allows to append a custom logger implementation.
Assuming that you want to stick to the default logging facility, customizing the log handler would then be as easy as droping a logging file and referencing it through the java.util.logging.config.file system property:
For example you can drop the logging configuration file under a config directory under the root path of your (fat) jar which may look as follows:
handlers = java.util.logging.MyFileHandler
config =
#...
You should then refrence that file in a system property as follows when starting your Vert.x application:
-Djava.util.logging.config.file=config/logging.properties
You can then access the Logger object in your classes as follows:
Logger logger = LoggerFactory.getLogger("some.package.MyClass");
Use that logger to log messages that will be handled by the configured handler(s):
logger.info("some informative message");
Note the use of a custom log handler in the properties file to emphasis the possibily of appending your own handler (which may extend the default FileHandler).
Check the Vert.x documentation for more informations on how to use explore the logging feature.
Most of loggers are async from the beginning , i.e. they are not write information immediately. Logs are stored into buffer, which is flushed by timeout or when it is full. So slf4j + log4 is good enough for most cases.
I need to split 1 batch application to 3 different. Code is almost the same, I have just modified ANT build script, and excluded or included some dependencies for different app. Than I have set different web.xml for each war. Each web.xml defines different spring application context with different beans for different behaviour.
All wars run on one tomcat server. Application used log4j, but now I refactored it to use slf4j instead. Thought I still need to use log4j under slf4j.
The problem I have is that each application log must appear in different log file,
even though class names are the same.
I can't write different log4j.properties file, because administrators placed it in tomcat/lib folder for all applications.
I have tried to place 3 files in tomcat/lib and change configuration file name for each application when initializing servlet, but it changed for all applications at same time.
Only solution I can think of now is to wrap log4j-over-slf4j, create 3 different slf4j log factories, that would append some prefix for each log name. For example, if I have this log:
private final Logger logger = LoggerFactory.getLogger(MainProcessor.class);
Each logging factory would genarate these logging names (with prefixes app1,app2 and app3) :
app1.com.test.MainProcessor
app2.com.test.MainProcessor
app3.com.test.MainProcessor
Is there any better way to deal with this problem ?
Try using a hook method, fire event, etc. so the logging doesn't happen in the class that is shared across the applications, but in some (top) class that is unique per application.
Variation is to statically access some logging class, use a singleton, etc. from the class where the logging should occur, but set context to that logging class on app initialization.
Executive Summary
How do I filter by the servlet in which the log message was invoked? (presently using 2.0 beta8)
Why on earth I would want to do that...
I have several existing web applications. They were written to rely on a proprietary logging system. I have re-implemented a key class from the proprietary system from scratch and added it as a class the proprietary system as a jar and log4j 2.0 as jars in tomcat, thereby utilizing the class loading load order in tomcat to divert the proprietary system into log4j. This succeeds and my log4j config now controls everything (Yay!).
But... (There's always a "But"!)
I was very pleased until I discovered that with all 4 applications deployed in the same container, they were not coordinating their writes to the single log file in the single configuration I had placed in conf/log4j2.xml (and specifed by passing -Dlog4j.configurationFile=/mnt/kui/tomcat/conf/log4j2.xml on the command line). I found some log messages with much earlier time stamps (hours earlier) in the middle of the log file. Out of order logs (and possibly overwritten log lines?) are not desirable of course.
I actually don't want them all in one file anyway and would prefer a log per application controlled by a single config file. Initially I thought this would be easy to achieve since log4j automatically sets up a LoggingContext with the name of the web application.
However I can't seem to find a filter implementation that will allow me to filter on the LoggingContext. I understand that from each application's perspective there is only one logging context (I think), but the same config file is read by 4 applications so from the config perspective LoggingContext is not unique.
I'm looking for a way to route each application to it's own file without having a config file for every application, or having to add classes to all the applications or edit war files (including web.xml). I'm sooo... close but It's not working.
Just to complicate matters, there is a jar file we wrote that is shared among all 4 applications that uses this logging too and one application has converted to using log4j directly in it's classes (but it still uses proprietary classes that reference the proprietary logging class that I replaced).
I have already seen http://logging.apache.org/log4j/2.x/manual/logsep.html and my case seems closest to '"Shared" Web Applications and REST Service Containers' but that case doesn't seem very well covered by that page.
You may want to look at the RoutingAppender which can be used to separate log files based on data in your ThreadContextMap. You could use the web app name as a unique key.
About the out of order logs, there was an issue with FastFileAppender in older betas. If append was false, the old file was not truncated but new log events would start to overwrite the old file from the beginning. (So after your most recent log event you would see yesterday's log events, for example). What version are you using?
I am planning to implement logging into a web application that I am currently working on but I am struggling with some of the details. What is the best way to go about logging a Java web application?
Specifically;
Where does the configuration file go in .war package file?
Where do people log to, relative or absolute path flat file, or a database?
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
Any other gotchas I should be aware of for web application logging?
Currently I am using Log4J but I imagine that the best practices would apply universally to all logging implementations.
EDIT:
One addition to the questions up top.
Where do you initilize the log
configuration?
In a traditional app, I do this at the entry point;
DOMConfigurator.configureAndWatch("log4j.xml");
What would the web application equivalent be?
I would recommend you to use SLF4J. This is simple logging facade which supports most of popular logging systems (Log4j, commons-logging, Java Logging API and Logback).
Using it, you will able to replace your underline logging system to any other, by simple CLASSPATH update.
The other benefit of SLF4J are parameterized calls, which reduces ugly logging code.
Actually, they recommends to use SLF4J with Logback.
Logback is a successor of Log4J. And it was designed by the same author.
Where does the configuration file go in .war package file?
Root of the classpath.
Where do people log to, relative or absolute path flat file, or a database?
Depends on the need. It's always better to use relative paths. Databases are good if you implement another application which will fetch logs from it and send them using email/sms
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
If you use Console appender, yes, it's going to be logged in your servlet container log file.
Any other gotchas I should be aware of for web application logging?
If you are logging from different threads use logback, it's thread-safe and it exposes parameterized log messages.
Logging to a DB adds another failure point. We had a situation where the prod servers logged to a DB and someone ran an expensive query on that DB that slowed it so much that the prod servers got very, very slow. Logging to a file can cause issues if you run out of space but it seems less likely to slow down the whole app.
I place my configuration on the default package: src/
and log to files using the ${catalina.home} system property:
log4j.appender.???.file=${catalina.home}/logs/system.log
It might be a good idea to place the config file somewhere where an admin can modify it without rebuilding your web app (e.g., so they can turn on detailed logging without waking you up in the middle of the night).
Unfortunately, there's no "official" way for locating externalized resources from a web app (correct me if I'm wrong). The most common way of doing it I've seen is to look through the directories in the classpath.
The excellent paper How to Do Application Logging Right has a bunch of gotchas.
I believe that your other questions have been answered by other people on this page.
I also recommend that you use SLF4J.
One last thing: having speakable representations of objects can save some time.
I recommend to call log API (log4j) via slf4j. Even if you use log4j, web container or depending modules may use different log API such as Java.util.logging or Jakarta commons logging. Slf4j provides bridge modules that redirect them to slf4j API. As a result, all log messages are written by log4j in that case.
put the log4j in the container (server) and create proper appenders per application
relative to server path, but that depends on your needs
we use appenders which log to different files, depends on your needs, e.g. one file for hibernate info/statistics, one for application only, etc.
don't log to much, it slows the application down
Personally I put the log4j.properties in the WEB-INF directory and use an init servlet with the following code :
public class InitServlet extends javax.servlet.http.HttpServlet implements javax.servlet.Servlet {
private static final String LOG4J_FILE = "WEB-INF/log4j.properties";
public InitServlet() {
super();
}
#Override
public void init() throws ServletException {
super.init();
PropertyConfigurator.configure(getServletContext().getRealPath(LOG4J_FILE));
LogFactory.getLog(InitServlet.class).info("LOG 4J configured");
}
}
Where does the configuration file go in .war package file?
At the root of the classpath but... Don't put the configuration file in the war package. You don't want to repackage and redeploy the application if you change the logging configuration, do you ? A better practice would be to put the configuration file somewhere in the classpath outside the war.
Where do people log to, relative or absolute path flat file, or a database?
I usually log to the file system on a separate partition (log files can grow very fast and should never block the application or the operating system if they become too big). I use most of time an absolute path based on the following model: /var/projects/<PROJECT_NAME>/<PRODUCT>/<CLUSTER_NAME>/logs/<INSTANCE_NAME>.log where <PROJECT_NAME> is the project name, <PRODUCT> can be Apache, Tomcat, Weblogic,..., <CLUSTER_NAME> the name of the cluster and <INSTANCE_NAME> the name of the instance inside the cluster. Logging to the file system is faster than in a database. The drawback is that logs aren't centralized if you are using several instances and physical machines. But merging can easily be done with a script.
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
Application server logs are application server logs, not application logs. Don't write to them but set up a logger tool (e.g. log4j) and write to application logs (understand dedicated).
Any other gotchas I should be aware of for web application logging?
If you are using log4j, don't forget to use the isDebugEnabled() before to log:
if(logger.isDebugEnabled()) {
logger.debug("Bla Bla Bla");
}
Where does the configuration file go in .war package file?
Usually, I do not place any logging configuration into the application, rather leaving that to the appserver admins to configure logging server-wide. In the rare cases I want the log4j configuration deployed with a webapp, WEB-INF is the usual path.
Where do people log to, relative or absolute path flat file, or a database?
Again, depends on appserver settings. One common log file for a appserver and rotating on a daily basis is the usual setup. If there are any app-specific needs, the admin may configure a separate logfile for an app (distinguished by package / class names).
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
See above. For tomcat used for development purposes, I'd just look for its logging (log4j) configuration and add app-specific specific there.
Any other gotchas I should be aware of for web application logging?
Performance. Limit the ,log level to a minimum (i.e. WARN or ERROR) once you go live. Use
if (log.isDebugEnabled()) { log.debug("..."); } and alike constructs in your code.
Note that if you just need a bit of logging, the servlet standard specifies that you can get the ServletContext and use the log methods there. That is the generic servlet equivalent of System.out.println.