I am deploying a Spring Boot 2.0.0-RC1 application as an init.d service, but I can't figure out how to configure the log rotation.
The app logs to /var/log/appname.log, but if I configure logrotate the logging stops after a rotation, because a new file is created, and the stdout/stderr redirection defined in the embedded script does not work anymore.
If I configure the log rotation in my logging system there are two problems: I can't create the files in /var/log, and I still have the redirection defined in the embedded script.
What is the proper solution for this?
I'm facing the same problem in several applications and adding copytruncate param is the solution because your Spring Boot application doesn’t understand that the file has changed (truncated) and is acting like a tail -f command (see How does the “tail” command's “-f” parameter work? for details).
Example:
/opt/payara41/glassfish/domains/domain1/logs/* {
daily
copytruncate
rotate 3
dateext
notifempty
}
copytruncate
Truncate the original log file to zero size in place after creating a copy, instead of moving the old log file and optionally creating a new one. It can be used when some program cannot be told to close its logfile and thus might continue writing (appending) to the previous log file forever.
Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log
file stays in place
I found the solution, it's the option copytruncate in logrotate.
My app deployed on Tomcat uses log4j to write a log file. If I delete that file, then the app does not recreate it. I also tried to recreate it manually, but it remains always empty. Is there any way to delete the log file (not from the app), create a new one in the same path with the same name, and that it can be written by the application?
Is there any way to delete the log file (not from the app), create a new one in the same path with the same name, and that it can be written by the application?
Nope. You need to get the application itself to restart logging.
The problem is that the log4j appender still has a handle for the deleted file, and will continue to write to it ... unaware that it has been deleted.
A better approach would be to have the application itself take care of "rotating" the logfile. Look at the classes that implement the log4j Appender interface for some ideas.
Using JBoss EAP 6.4 (AS 7.x I guess).
By default, JBoss' logging service is capturing all application output to stdout (and presumably stderr) and wrapping it in its own log4j-based logs. When running locally I want to totally disable this (annoying) feature1, but the references I've found on the Interwebs all either don't work or are for older versions of JBoss. I've tried excluding every possible logging framework in the jboss-deployment-structure configuration, passing -Dorg.jboss.logging.per-deployment=false as a system property, but still JBoss captures stdout.
How can I disable it in this version of JBoss?
[1] If you must know the reason, it's because we have detailed logging configuration via logback and when running locally in an IDE want to be able to see and control that log output in the console without JBoss' logging service getting in the way.
It's hard-coded in the entry points to capture stdout and stderr. This is done so both streams are written to the defined log handlers. Because of this there is no real clean way around it. However there are ways to make it at least look a little better.
You can create a new console-handler and define a stdout logger to ensure only the simple message is written though.
Here are some CLI commands to configure a logger named stdout to just print the message it receives.
/subsystem=logging/pattern-formatter=plain-formatter:add(pattern="%s")
/subsystem=logging/console-handler=plain-console:add(autoflush=true, target=System.out, named-formatter=plain-formatter)
/subsystem=logging/logger=stdout:add(use-parent-handlers=false, handlers=[plain-console])
Note: It could be my test logback.xml configuration, but I had to use a pattern of %s%n for the plain-formatter.
The only other option I can think of would be to write your own logback ConsoleAppender that creates an output stream based on java.io.FileOutputStream.out rather than using System.out.
I am developing an Eclipse RCP application and have gone to some pains to get log4j2 to work within the app. All seems to work fine now, and as a finishing touch I wanted to make all loggers asynchronously.
I've managed to get the LMAX Disruptor on the classpath, and think I've solved the issue of providing sun.misc as well. Set the VM argument -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector in the run config and set up log4j2.xml file correctly as well. I think. And that's where the problem is. I'd like to be able to verify that my application logs asynchronously in the proper fashion, so I can enjoy the benefits latency-wise.
How can I - then - verify that my loggers are working asynchronously, utilising the LMAX Dirsuptor in the process?
There are two types of async logger, handled by different classes.
All loggers async: the AsyncLogger class - activated when you use AsyncLoggerContextSelector
Mixing sync with async loggers: the AsyncLoggerConfig class - when your configuration file has <AsyncRoot> or <AsyncLogger> elements nested in the configuration for <Loggers>.
In your case you are making all loggers async, so you want to put your breakpoint in AsyncLogger#logMessage(String, Level, Marker, Message, Throwable).
Another way to verify is by setting <Configuration status="trace"> at the top of your configuration file. This will output internal log4j log messages on log4j is configured. You should see something like "Starting AsyncLogger disruptor...". If you see this all loggers are async.
Put a breakpoint in org.apache.logging.log4j.core.async.AsyncLoggerConfig#callAppenders. Then you can watch as the event is put into the disruptor. Likewise org.apache.logging.log4j.core.config.LoggerConfig#callAppenders should be getting hit for synchronous logging OR getting hit from the other side of the disruptor for async logging (at which point everything is synchronous again).
I am planning to implement logging into a web application that I am currently working on but I am struggling with some of the details. What is the best way to go about logging a Java web application?
Specifically;
Where does the configuration file go in .war package file?
Where do people log to, relative or absolute path flat file, or a database?
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
Any other gotchas I should be aware of for web application logging?
Currently I am using Log4J but I imagine that the best practices would apply universally to all logging implementations.
EDIT:
One addition to the questions up top.
Where do you initilize the log
configuration?
In a traditional app, I do this at the entry point;
DOMConfigurator.configureAndWatch("log4j.xml");
What would the web application equivalent be?
I would recommend you to use SLF4J. This is simple logging facade which supports most of popular logging systems (Log4j, commons-logging, Java Logging API and Logback).
Using it, you will able to replace your underline logging system to any other, by simple CLASSPATH update.
The other benefit of SLF4J are parameterized calls, which reduces ugly logging code.
Actually, they recommends to use SLF4J with Logback.
Logback is a successor of Log4J. And it was designed by the same author.
Where does the configuration file go in .war package file?
Root of the classpath.
Where do people log to, relative or absolute path flat file, or a database?
Depends on the need. It's always better to use relative paths. Databases are good if you implement another application which will fetch logs from it and send them using email/sms
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
If you use Console appender, yes, it's going to be logged in your servlet container log file.
Any other gotchas I should be aware of for web application logging?
If you are logging from different threads use logback, it's thread-safe and it exposes parameterized log messages.
Logging to a DB adds another failure point. We had a situation where the prod servers logged to a DB and someone ran an expensive query on that DB that slowed it so much that the prod servers got very, very slow. Logging to a file can cause issues if you run out of space but it seems less likely to slow down the whole app.
I place my configuration on the default package: src/
and log to files using the ${catalina.home} system property:
log4j.appender.???.file=${catalina.home}/logs/system.log
It might be a good idea to place the config file somewhere where an admin can modify it without rebuilding your web app (e.g., so they can turn on detailed logging without waking you up in the middle of the night).
Unfortunately, there's no "official" way for locating externalized resources from a web app (correct me if I'm wrong). The most common way of doing it I've seen is to look through the directories in the classpath.
The excellent paper How to Do Application Logging Right has a bunch of gotchas.
I believe that your other questions have been answered by other people on this page.
I also recommend that you use SLF4J.
One last thing: having speakable representations of objects can save some time.
I recommend to call log API (log4j) via slf4j. Even if you use log4j, web container or depending modules may use different log API such as Java.util.logging or Jakarta commons logging. Slf4j provides bridge modules that redirect them to slf4j API. As a result, all log messages are written by log4j in that case.
put the log4j in the container (server) and create proper appenders per application
relative to server path, but that depends on your needs
we use appenders which log to different files, depends on your needs, e.g. one file for hibernate info/statistics, one for application only, etc.
don't log to much, it slows the application down
Personally I put the log4j.properties in the WEB-INF directory and use an init servlet with the following code :
public class InitServlet extends javax.servlet.http.HttpServlet implements javax.servlet.Servlet {
private static final String LOG4J_FILE = "WEB-INF/log4j.properties";
public InitServlet() {
super();
}
#Override
public void init() throws ServletException {
super.init();
PropertyConfigurator.configure(getServletContext().getRealPath(LOG4J_FILE));
LogFactory.getLog(InitServlet.class).info("LOG 4J configured");
}
}
Where does the configuration file go in .war package file?
At the root of the classpath but... Don't put the configuration file in the war package. You don't want to repackage and redeploy the application if you change the logging configuration, do you ? A better practice would be to put the configuration file somewhere in the classpath outside the war.
Where do people log to, relative or absolute path flat file, or a database?
I usually log to the file system on a separate partition (log files can grow very fast and should never block the application or the operating system if they become too big). I use most of time an absolute path based on the following model: /var/projects/<PROJECT_NAME>/<PRODUCT>/<CLUSTER_NAME>/logs/<INSTANCE_NAME>.log where <PROJECT_NAME> is the project name, <PRODUCT> can be Apache, Tomcat, Weblogic,..., <CLUSTER_NAME> the name of the cluster and <INSTANCE_NAME> the name of the instance inside the cluster. Logging to the file system is faster than in a database. The drawback is that logs aren't centralized if you are using several instances and physical machines. But merging can easily be done with a script.
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
Application server logs are application server logs, not application logs. Don't write to them but set up a logger tool (e.g. log4j) and write to application logs (understand dedicated).
Any other gotchas I should be aware of for web application logging?
If you are using log4j, don't forget to use the isDebugEnabled() before to log:
if(logger.isDebugEnabled()) {
logger.debug("Bla Bla Bla");
}
Where does the configuration file go in .war package file?
Usually, I do not place any logging configuration into the application, rather leaving that to the appserver admins to configure logging server-wide. In the rare cases I want the log4j configuration deployed with a webapp, WEB-INF is the usual path.
Where do people log to, relative or absolute path flat file, or a database?
Again, depends on appserver settings. One common log file for a appserver and rotating on a daily basis is the usual setup. If there are any app-specific needs, the admin may configure a separate logfile for an app (distinguished by package / class names).
Does Log4J logging go directly into the application server log file automatically or is that something you have to set up? In this case I am using Tomcat, but I often use Jrun.
See above. For tomcat used for development purposes, I'd just look for its logging (log4j) configuration and add app-specific specific there.
Any other gotchas I should be aware of for web application logging?
Performance. Limit the ,log level to a minimum (i.e. WARN or ERROR) once you go live. Use
if (log.isDebugEnabled()) { log.debug("..."); } and alike constructs in your code.
Note that if you just need a bit of logging, the servlet standard specifies that you can get the ServletContext and use the log methods there. That is the generic servlet equivalent of System.out.println.