We Keep Coding

How to get log4j2.properties file to read filename from pom.xml in Spring Boot

I been searching internet and i might have missed. But what i am trying to achieve in my code is, have a log4j2.properties file read file names defined in pom.xml in a Spring boot project. Something like below.
pom.xml:
<properties>
<log.file>/expo/net/logs/xol/aws.log</log.file>
<status.file>/export/net/logs/xol/tdlg.log</status.file>
</properties>
log4j2.properties:
appender.main.type=RollingFile
appender.main.name=MAIN
#appender.main.fileName=${log.file}
appender.main.filePattern=${log.file}.%d{yyyyMMddHH}
appender.main.layout.type=PatternLayout
appender.main.layout.pattern=%d{MM/dd/yyyy HH:mm:ss.SSS} %-5p %highlight{%t} %replace{%msg}{\n\r|\n|\r}{ }%n
appender.main.policies.type=Policies
appender.main.policies.time.type=TimeBasedTriggeringPolicy
appender.main.policies.time.interval=1
appender.main.policies.time.modulate=true
My expectation is when run the maven correct values should get replaced in.
Any suggestions where i am going wrong?

I think i found the solution. Someone had asked similar question.
Maven resource filtering not working - because of spring boot dependency
It is explained on below link
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html
In my log4j2.properties file i replaced "${" and "}" with "#".
appender.main.type=RollingFile
appender.main.name=MAIN
#appender.main.fileName=#log.file#
appender.main.filePattern=#log.file#.%d{yyyyMMddHH}
appender.main.layout.type=PatternLayout
appender.main.layout.pattern=%d{MM/dd/yyyy HH:mm:ss.SSS} %-5p %highlight{%t} %replace{%msg}{\n\r|\n|\r}{ }%n
appender.main.policies.type=Policies
appender.main.policies.time.type=TimeBasedTriggeringPolicy
appender.main.policies.time.interval=1
appender.main.policies.time.modulate=true
Worked like a charm. Thanks guys for all the inputs.

How to lookup property parameter in web app for log4j2?

I want to configure log4j2 to lookup the logging path dynamically from web startup (tomcat).
From the docs (http://logging.apache.org/log4j/2.x/manual/lookups.html) there is a web: lookup with different parameter possibilities.
At first I'm trying the provided example:
<Appenders>
<File name="ApplicationLog" fileName="${web:rootDir}/app.log"/>
</Appenders>
Result: ERROR Unable to create file ${web:rootDir}/app.log java.io.IOException
I also tried the other buildin properties like servletContextName and contextPath with the same error message.
So I'm probably still missing something important. But what?

Add log4j-web JAR
Your usual log4j-core JAR file on its own is not enough.
There is an additional component called log4j-web.
For automatic replacement of the ${web:...} placeholders in your logging configuration, you need the additional log4j-web JAR file. If you don't have this log4j-web JAR file, then Log4j2 itself will still work, but it will not replace these placeholders with anything. So for example ${web:rootDir} will just end up as the literal text ${web:rootDir}.
Perhaps you are missing that dependency in your project?
Log4j-web on Maven Central

Is ${web:rootDir} a value? The placeholder is not replaced by its real value.
In the page you can red:
Information on how to use Lookups in configuration files can be found in the Property Substitution: http://logging.apache.org/log4j/2.x/manual/configuration.html#PropertySubstitution

How to Change log4j Configuration When Packaging

I have a fairly simple Java application that is being distributed out to users as an executable JAR file. In order to track usage and assist in diagnostics, I want to add logging capability to the application but, from one execution to the next, I can never be sure where the users are going to be running this application.
In order to make this work, I have configured my application to use a log4j socket appender - no matter where the user is running this application, messages are sent to the socket and properly logged, which is perfect.
What I'm running into now, though, it that I would prefer to use different log4j configurations when I am running locally vs. when I deploy this application into production. When running locally (from my IDE, IntelliJ), I would prefer to use a console appender for a couple reasons:
It's trivial to see what is happening.
Should I be developing without an Internet connection, I don't want to hang up on not being able to connect.
I don't want to clutter the production logs with what I am doing during development.
When I package the application and distribute it, however, I would like it to use the socket appender, rather than the console appender.
Is there any easy way to accomplish what I want? I am using Maven to build my application, but I am not horribly skilled with it.

Here's a simpler approach to your problem...
Put two log4j.properties files under src/main/resources directory, like this:-
src/main/resources
|- log4j.properties
|- log4j_dev.properties
log4j.properties contains the production configuration, and by default, Log4J will automatically pick this file up because you are using the standard file name.
log4j_dev.properties contains the development configuration. To ensure IntelliJ picks this up during development, you will overwrite the log configuration using -Dlog4j.configuration=<configuration-file> VM option. I attached a screenshot on how you might configure this using IntelliJ:-

This turned out to be simpler than I originally expected - and Vikdor's comment led me to the right answer. I had started digging into having multiple profiles (one for dev and one for prod) but later realized I didn't really need to have two. When working locally, I wasn't really going through the Maven build process - I was just executing the Java classes. The only time I went through the Maven build process was when I wanted to package my application for deployment. As such, I simply added a section to my POM file that uses the maven-antrun-plugin. My final code ended up looking like this:
File: log4j.properties:
# Set root logger level to DEBUG and its only appender to A1.
log4j.rootLogger=DEBUG, A1
# A1 is set to be a ConsoleAppender.
log4j.appender.A1=org.apache.log4j.ConsoleAppender
# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
File: log4j_production.properties
# Set root logger level to DEBUG and its only appender to A1.
log4j.rootLogger=DEBUG, A2
log4j.appender.A2=org.apache.log4j.net.SocketAppender
log4j.appender.A2.remoteHost=my.server.com
log4j.appender.A2.port=6000
log4j.appender.A2.layout=org.apache.log4j.PatternLayout
log4j.appender.A2.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
File: POM.xml
...
<build>
<finalName>${project.artifactId}</finalName>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<delete file="${project.build.outputDirectory}/log4j.properties"/>
<copy file="src/main/resources/log4j_production.properties"
toFile="${project.build.outputDirectory}/log4j.properties"/>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
...
That seemed to do the trick. When I run locally, I load log4j.properties, which gives me a console appender. When I package for a production distribution, Maven will replace the default log4j.properties file with the alternate one, which uses a socket appender.
Thanks!

I will strongly recommend making it determined in run-time instead of build-time.
For example, when you are running your app, have config/ directory put in beginning of classpath, and put your log4j.xml there. Of course, you may still keep a default log4.xml in your application JAR, so that if you don't provide it in deployment, your app can still run.

Tomcat App says "Unable to configure the logging system. No log.properties was found."

I have an embedded tomcat app and whenever I start it up I see this error printed to the console twice:
Unable to configure the logging system. No log.properties was found.
It seems stupid, but I've done some googling and searched stackoverflow and can't seem to find someone experiencing this problem.
My main class Looks roughly like this:
public class AuthServerEntryPoint {
static {
org.apache.log4j.PropertyConfigurator.configure("conf/log4j.properties");
}
public static void main(String[] args) {
// ...
}
"conf/log4j.properties" contains a seemingly valid configuration:
log4j.appender.mainAppend=org.apache.log4j.ConsoleAppender
log4j.appender.mainAppend.layout=org.apache.log4j.PatternLayout
log4j.appender.mainAppend.layout.ConversionPattern=%d %p [%t] %c -- %m%n
log4j.appender.fileAppend=org.apache.log4j.FileAppender
log4j.appender.fileAppend.layout=org.apache.log4j.PatternLayout
log4j.appender.fileAppend.layout.ConversionPattern=%d %p [%t] %c - %m%n
log4j.appender.fileAppend.file=logs/myservice.log
log4j.rootLogger = info, fileAppend
log4j.logger.com.mycompany.myservice = debug, fileAppend
And the logging actually does work - i.e., logs are correctly written to myservice.log. So what gives?
Thanks!
-Carl

By embedded Tomcat app, do you mean that you are starting Tomcat from Java code and are using the class org.apache.catalina.startup.Embedded?
If yes, my guess is that Tomcat might not be able to find its logging configuration file that is set up in catalina.sh (or equivalent) when using the scripts:
LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
The odd part is that this file is called logging.properties, not log.properties, so I'm really not sure. I didn't check Tomcat sources though, maybe they are doing some kind of black magic in there.
Could you just try to rename $CATALINA_BASE/conf/logging.properties into log.properties (or not) and to put it on the classpath of your app (or to set -Djava.util.logging.config.file)?

By default, log4j will look for a logging properties file on the classpath. Put your webapp's logging properties config file into its WEB-INF/classes directory; e.g.
$CATALINA_HOME/webapps/<yourApp>/WEB-INF/classes/log4j.properties
This is the simple approach to getting a webapp to use Log4j is recommended by the referenced documentation. And it is the approach I use myself for webapp logging.
There are various other ways to configure Log4j logging on Tomcat as well. It is possible that your Tomcat has been (partly) configured to use one of them, but something has not been done quite right.
Configuring Tomcat' log4j logging via the system properties is an option that avoids figuring out where log4j is looking ... and what is going wrong. But you are then stuck with creating/using a custom launch script or having to remember to set environment variables before launching Tomcat.
References:
Configuring logging on Tomcat 5.5
Configuring logging on Tomcat 6
The "Default Initialization under Tomcat" section of the Log4j Manual

Log4j, configuring a Web App to use a relative path

I have a java webapp that has to be deployed on either Win or Linux machines. I now want to add log4j for logging and I'd like to use a relative path for the log file as I don't want to change the file path on every deployment. The container will most likely be Tomcat but not necessarily.
What's the best way of doing this?

Tomcat sets a catalina.home system property. You can use this in your log4j properties file. Something like this:
log4j.rootCategory=DEBUG,errorfile
log4j.appender.errorfile.File=${catalina.home}/logs/LogFilename.log
On Debian (including Ubuntu), ${catalina.home} will not work because that points at /usr/share/tomcat6 which has no link to /var/log/tomcat6. Here just use ${catalina.base}.
If your using another container, try to find a similar system property, or define your own. Setting the system property will vary by platform, and container. But for Tomcat on Linux/Unix I would create a setenv.sh in the CATALINA_HOME/bin directory. It would contain:
export JAVA_OPTS="-Dcustom.logging.root=/var/log/webapps"
Then your log4j.properties would be:
log4j.rootCategory=DEBUG,errorfile
log4j.appender.errorfile.File=${custom.logging.root}/LogFilename.log

I've finally done it in this way.
Added a ServletContextListener that does the following:
public void contextInitialized(ServletContextEvent event) {
ServletContext context = event.getServletContext();
System.setProperty("rootPath", context.getRealPath("/"));
}
Then in the log4j.properties file:
log4j.appender.file.File=${rootPath}WEB-INF/logs/MyLog.log
By doing it in this way Log4j will write into the right folder as long as you don't use it before the "rootPath" system property has been set. This means that you cannot use it from the ServletContextListener itself but you should be able to use it from anywhere else in the app.
It should work on every web container and OS as it's not dependent on a container specific system property and it's not affected by OS specific path issues.
Tested with Tomcat and Orion web containers and on Windows and Linux and it works fine so far.
What do you think?

If you use Spring you can:
1) create a log4j configuration file, e.g. "/WEB-INF/classes/log4j-myapp.properties"
DO NOT name it "log4j.properties"
Example:
log4j.rootLogger=ERROR, stdout, rollingFile
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - <%m>%n
log4j.appender.rollingFile=org.apache.log4j.RollingFileAppender
log4j.appender.rollingFile.File=${myWebapp-instance-root}/WEB-INF/logs/application.log
log4j.appender.rollingFile.MaxFileSize=512KB
log4j.appender.rollingFile.MaxBackupIndex=10
log4j.appender.rollingFile.layout=org.apache.log4j.PatternLayout
log4j.appender.rollingFile.layout.ConversionPattern=%d %p [%c] - %m%n
log4j.appender.rollingFile.Encoding=UTF-8
We'll define "myWebapp-instance-root" later on point (3)
2) Specify config location in web.xml:
<context-param>
<param-name>log4jConfigLocation</param-name>
<param-value>/WEB-INF/classes/log4j-myapp.properties</param-value>
</context-param>
3) Specify a unique variable name for your webapp's root, e.g. "myWebapp-instance-root"
<context-param>
<param-name>webAppRootKey</param-name>
<param-value>myWebapp-instance-root</param-value>
</context-param>
4) Add a Log4jConfigListener:
<listener>
<listener-class>org.springframework.web.util.Log4jConfigListener</listener-class>
</listener>
If you choose a different name, remember to change it in log4j-myapp.properties, too.
See my article (Italian only... but it should be understandable):
http://www.megadix.it/content/configurare-path-relativi-log4j-utilizzando-spring
UPDATE (2009/08/01)
I've translated my article to English:
http://www.megadix.it/node/136

Just a comment on Iker's solution.
ServletContext is a good solution for your problem. But I don't think it is good for maintains. Most of the time log files are required to be saved for long time.
Since ServletContext makes the file under the deployed file, it will be removed when server is redeployed. My suggest is to go with rootPath's parent folder instead of child one.

Doesn't log4j just use the application root directory if you don't specify a root directory in your FileAppender's path property? So you should just be able to use:
log4j.appender.file.File=logs/MyLog.log
It's been awhile since I've done Java web development, but this seems to be the most intuitive, and also doesn't collide with other unfortunately named logs writing to the ${catalina.home}/logs directory.

As a further comment on https://stackoverflow.com/a/218037/2279200 - this may break, if the web app implicitely starts other ServletContextListener's, which may get called earlier and which already try to use log4j - in this case, the log4j configuration will be read and parsed already before the property determining the log root directory is set => the log files will appear somewhere below the current directory (the current directory when starting tomcat).
I could only think of following solution to this problem:
- rename your log4j.properties (or logj4.xml) file to something which log4j will not automatically read.
- In your context filter, after setting the property, call the DOM/PropertyConfigurator helper class to ensure that your log4j-.{xml,properties} is read
- Reset the log4j configuration (IIRC there is a method to do that)
This is a bit brute force, but methinks it is the only way to make it watertight.

In case you're using Maven I have a great solution for you:
Edit your pom.xml file to include following lines:
<profiles>
<profile>
<id>linux</id>
<activation>
<os>
<family>unix</family>
</os>
</activation>
<properties>
<logDirectory>/var/log/tomcat6</logDirectory>
</properties>
</profile>
<profile>
<id>windows</id>
<activation>
<os>
<family>windows</family>
</os>
</activation>
<properties>
<logDirectory>${catalina.home}/logs</logDirectory>
</properties>
</profile>
</profiles>
Here you define logDirectory property specifically to OS family.
Use already defined logDirectory property in log4j.properties file:
log4j.appender.FILE=org.apache.log4j.RollingFileAppender
log4j.appender.FILE.File=${logDirectory}/mylog.log
log4j.appender.FILE.MaxFileSize=30MB
log4j.appender.FILE.MaxBackupIndex=10
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d{ISO8601} [%x] %-5p [%t] [%c{1}] %m%n
That's it!
P.S.: I'm sure this can be achieved using Ant but unfortunately I don't have enough experience with it.

My suggestion is the log file should always be logged above the root context of the webApp, so in case we redeploy the webApp, we don't want to override the existing log files.

My solution is similar to Iker Jimenez's solution, but instead of using System.setProperty(...) I use org.apache.log4j.PropertyConfigurator.configure(Properties). For that I also need log4j to be unable to find its configuration on its own and I load it manually (both points described in Wolfgang Liebich's answer).
This works for Jetty and Tomcat, standalone or run from IDE, requires zero configuration, allows to put each app's logs in their own folder, no matter how many apps inside the container (which is the problem with the System-based solution). This way one can also put the log4j config file anywhere inside the web app (e.g. in one project we had all config files inside WEB-INF/).
Details:
I have my properties in the log4j-no-autoload.properties file in the classpath (e.g. in my Maven project it's originally in src/main/resources, gets packaged into WEB-INF/classes),
It has the file appender configured as e.g.:
log4j.appender.MyAppFileAppender = org.apache.log4j.FileAppender
log4j.appender.MyAppFileAppender.file = ${webAppRoot}/WEB-INF/logs/my-app.log
...
And I have a context listener like this (gets much shorter with Java 7's "try-with-resource" syntax):
#WebListener
public class ContextListener implements ServletContextListener {
#Override
public void contextInitialized(final ServletContextEvent event) {
Properties props = new Properties();
InputStream strm =
ContextListener.class.getClassLoader()
.getResourceAsStream("log4j-no-autoload.properties");
try {
props.load(strm);
} catch (IOException propsLoadIOE) {
throw new Error("can't load logging config file", propsLoadIOE);
} finally {
try {
strm.close();
} catch (IOException configCloseIOE) {
throw new Error("error closing logging config file", configCloseIOE);
}
}
props.put("webAppRoot", event.getServletContext().getRealPath("/"));
PropertyConfigurator.configure(props);
// from now on, I can use LoggerFactory.getLogger(...)
}
...
}

You can specify relative path to the log file, using the work directory:
appender.file.fileName = ${sys:user.dir}/log/application.log
This is independent from the servlet container and does not require passing custom variable to the system environment.