We Keep Coding

Providing a method name in the log4j2 GelfLayout

I'm trying to include a calling method name as a separate field in the Gelf:
<Socket name="Graylog" protocol="udp" host="127.0.0.1" port="12201">
<GelfLayout>
<KeyValuePair key="method" value="%M"/>
</GelfLayout>
</Socket>
But in the Graylog this field's value appears just as %M, no value substitution is happening.
I read the manual, but couldn't find anything about resolving message pattern placeholders in the custom fields.
I know that logstash-gelf can do this, but is it possible to do this without using an extra plugin, using the standard log4j2?

I wanted to show what ID the thread had, but just as you, I noticed that the text was treated as static text. I looked into this and acording to the documentation it should be possible to use patterns, but not in <KeyValuePair>
<GelfLayout includeThreadContext="true" threadContextIncludes="loginId,requestId">
%d %5p [%t] %c{1} %X{loginId, requestId} - %m%n
<KeyValuePair key="additionalField1" value="constant value"/>
<KeyValuePair key="additionalField2" value="$${ctx:key}"/>
</GelfLayout>
Look at the second row, above. It looks as if it would be possible to use pattern placeholders as a general message. I have, however, not been able to get this to work, maybe the version I use is to old.
I have found out that Log4j2 does not currently support what we want. But it will do in the next version (2.15.0). I have no idea, how ever, when the next version will be released, se issue LOG4J2-3041 for details.
References:
https://logging.apache.org/log4j/2.x/manual/layouts.html#GELFLayout
https://issues.apache.org/jira/browse/LOG4J2-3041

log4j Appender additivity for custom log level

I have a scenario where I have a separate custom level defined XYZLogLevel for logging and I have 2 rolling file appenders where 2nd appender is specifically reserved to log log messages from XYZLogLevel. However the logs are going in both the files which is undesirable.
Note:-
Logs are not package specific, so adding additivity for package wont
work.
Everything has to be done through log4j.properties file.
Adding LevelRangeFilter to first appender partially resolves it, but
when I add 3rd appender for another custom level, then I again see
duplication in 2nd and 3rd appender.
log4j.rootCategory=ERROR, F, XYZLOG, LMNLOG
log4j.appender.F=org.apache.log4j.RollingFileAppender
log4j.appender.F.File=f_log.log
log4j.appender.F.MaxFileSize=5MB
log4j.appender.F.MaxBackupIndex=10
log4j.appender.F.layout = org.apache.log4j.PatternLayout
log4j.appender.F.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss.SSS} %c{1} [%p] %m%n
log4j.appender.XYZLOG=org.apache.log4j.RollingFileAppender
log4j.appender.XYZLOG.File=xyz_reporting.log
log4j.appender.XYZLOG.threshold=XYZLOG#com.services.domain.xyzlogs.XYZLogLevel
log4j.appender.XYZLOG.filter.a=org.apache.log4j.varia.LevelMatchFilter
log4j.appender.XYZLOG.filter.a.LevelToMatch=XYZLOG#com.services.domain.xyzlogs.XYZLogLevel
log4j.appender.XYZLOG.filter.a.AcceptOnMatch=true
log4j.appender.XYZLOG.MaxFileSize=100KB
log4j.appender.XYZLOG.MaxBackupIndex=10
log4j.appender.XYZLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.XYZLOG.layout.ConversionPattern=%m%n
log4j.appender.LMNLOG=org.apache.log4j.RollingFileAppender
log4j.appender.LMNLOG.File=lmn_reporting.log
log4j.appender.LMNLOG.threshold=LMNLOG#com.services.domain.lmnlogs.LMNLogLevel
log4j.appender.LMNLOG.filter.a=org.apache.log4j.varia.LevelMatchFilter
log4j.appender.LMNLOG.filter.a.LevelToMatch=LMNLOG#com.services.domain.lmnlogs.LMNLogLevel
log4j.appender.LMNLOG.filter.a.AcceptOnMatch=true
log4j.appender.LMNLOG.MaxFileSize=100KB
log4j.appender.LMNLOG.MaxBackupIndex=10
log4j.appender.LMNLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.LMNLOG.layout.ConversionPattern=%m%n
Maybe this can be solved through org.apache.log4j.varia.DenyAllFilter but somehow I was not able to configure that for properties file. I believe that functionality is only for XML configuration.
Its a very tricky sitution as I dont have the liberty to switch to XML configuration, any help on this would be appreciated guys.
Link and examples would be great as I understand them quickly.

I am answering my own question.
First you need to create a logger and set additivity as FALSE.
log4j.logger.XZYLOG=XZYLOG#XYZLOG#com.services.domain.xyzlogs.XYZLogLevel, XYZAPPENDER
log4j.additivity.XZYLOG=false
log4j.logger.LMNLOG=LMNLOG#XYZLOG#com.services.domain.lmnlogs.LMNLogLevel, LMNAPPENDER
log4j.additivity.LMNLOG=false
Then configure the appenders in following way.
log4j.appender.XYZLOGAPPENDER=org.apache.log4j.RollingFileAppender
log4j.appender.XYZLOGAPPENDER.File=xyz_reporting.log
log4j.appender.XYZLOGAPPENDER.threshold=XYZLOG#com.services.domain.xyzlogs.XYZLogLevel
log4j.appender.XYZLOGAPPENDER.MaxFileSize=100KB
log4j.appender.XYZLOGAPPENDER.MaxBackupIndex=10
log4j.appender.XYZLOGAPPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.XYZLOGAPPENDER.layout.ConversionPattern=%m%n
log4j.appender.LMNLOGAPPENDER=org.apache.log4j.RollingFileAppender
log4j.appender.LMNLOGAPPENDER.File=lmn_reporting.log
log4j.appender.LMNLOGAPPENDER.threshold=LMNLOG#com.services.domain.lmnlogs.LMNLogLevel
log4j.appender.LMNLOGAPPENDER.MaxFileSize=100KB
log4j.appender.LMNLOGAPPENDER.MaxBackupIndex=10
log4j.appender.LMNLOGAPPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.LMNLOGAPPENDER.layout.ConversionPattern=%m%n
You may or may not need the filter in this case.
Note: I cannot post company code here thats why I modified the solution.
This is to give you general idea how I have solved this problem.

Log4j2 Rolling appender - fileName "sliding" according to pattern

I am looking for rollover strategy where current log (active output target in manual's terminology) file name is not fixed but specified by a pattern, or - more precisely - same pattern as in filePattern attribute.
I want to achieve daily rollover where today's log is, say, log-2015-05-05.log and on midnight framework just stops writing it and starts writing into log-2015-05-06.log. However, AFAIK, current configuration allows only
<RollingFile name="ROLFILE"
fileName="log.log"
filePattern="log-%d{yyyy-MM-dd}.log"
>
Specifying same value into fileName attribute doesn't work (leads to file with sensitive characters literally interpreted). I noticed no example or SO question with such a dynamic value of fileName. Note the fileName="log-${date:yyyy-MM-dd}.log" doesn't solve problem since expression is evaluated only at startup and events are still sent into file even if their timestamp doesn't match the expression.
I am migrating from Log4j 1.2 to Log4j 2.2. In old version, required behavior was possible using
<appender name="ROLFILE" class="org.apache.log4j.rolling.RollingFileAppender">
<rollingPolicy class="org.apache.log4j.rolling.TimeBasedRollingPolicy">
<param name="FileNamePattern" value="log-%d{yyyy-MM-dd}.log" />
</rollingPolicy>
...
I prefer to preserve current way since some log analyzing tools rely on it.
Is it possible in Log4j2?
Thanks.

Note sure if this works, but you can try using a double $$ in the date lookup. This allows the variable to be resolved at runtime.
<RollingFile name="ROLFILE"
fileName="log-$${date:yyyy-MM-dd}.log"
filePattern="oldlog-%d{yyyy-MM-dd}.log"
>
You may need to be careful to ensure that the active output target file name is different from the after-rollover file name. (I used 'oldlog' in the snippet above.)

Finally, I wrote my own rollover strategy which generates different set of rollover actions. Instead renaming active file the active file name is simply replaced inside RollingFileManager. Yes, it's ugly reflection hack and also appender must be initialized with fileName constant corresponding with current date and having same pattern, e.g.
<RollingFile name="ROLFILE"
fileName="log-${date:yyyy-MM-dd}.log"
filePattern="log-%d{yyyy-MM-dd}.log"
>
<SlidingFilenameRolloverStrategy />
...
yet for me this solution is worth doing it despite these small drawbacks.
(Initial fileName stays forever as a key in AbstractManager registry MAP even if in manager itself it has been changed - seems it doesn't mind, I also tried replacing manager in registry for new one but it's impossible to collect all parameters necessary for its construction.)
I hope this hack shouldn't have been so ugly if RollingFileManager API made it possible normal way. I got some hope seeing this javadoc but framework AFAIK never utilizes this field, let alone for mutating RollingFileAppender.

I think it would work just fine using:
fileName="log-${date:yyyy-MM-dd}.log"
filePattern="log-%d{yyyy-MM-dd}.log"
I use it with log4j2 version 2.5

This has been implemented in Log4j 2.8 (see issue LOG4J2-1101).
Currently it only works with a RollingFile appender by omitting the filename parameter and using a DirectWriteRolloverStrategy.
Also, this feature seems to have some issues with the TimeBasedTriggeringPolicy (the first rollover doesn't happen so every logfile is offset by one interval); CronTriggeringPolicy works properly.
Example config:
<RollingRandomAccessFile name="MyLogger"
filePattern="logs/application.%d{yyyy-MM-dd}.log">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
<Policies>
<CronTriggeringPolicy schedule="0 * * * * ?" evaluateOnStartup="true"/>
</Policies>
<DirectWriteRolloverStrategy/>
</RollingRandomAccessFile>
Support for RollingRandomAccessFile appender is requested in issue LOG4J2-1878.
Edit: Changed to CronTriggeringPolicy policy after finding TimeBasedTriggeringPolicy had issues.

Log4j: Conditional additivity (or propagate event to root logger based on condition?)

I've very simple task: write all events (warn and above) to main log and for certain logger write debug only messages to second log file (warn and above must be logged to main). Sometimes I wish to see debug messages in main log as well (e.g. log4j.logger.com.test=debug)
Besides I must use log4j properties file syntax. I don't understand if additivity (see my last line) can be conditional in that case or should I use complete different approach for the task (which one?).
What I've for now:
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.EnhancedPatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %-5p [%t] %c:%L - %m%n
log4j.rootLogger=warn, stdout
log4j.appender.file2=org.apache.log4j.RollingFileAppender
log4j.appender.file2.maxFileSize=10240KB
log4j.appender.file2.maxBackupIndex=30
log4j.appender.file2.File=${catalina.home}/logs/test.log
log4j.appender.file2.encoding=UTF-8
log4j.appender.file2.layout=org.apache.log4j.PatternLayout
log4j.appender.file2.layout.ConversionPattern=%d %-5p [%t] %c:%L - %m%n
log4j.logger.org.hibernate = debug, file2
log4j.additivity.org.hibernate = false
log4j.logger.com.test=debug

I would recommend you look at Log4J2 as log4j1.x is no longer actively maintained (and Log4J2 has several other advantages like performance improvements).
A common use case is writing debug and higher (info,warn, error, fatal) to a file for developers and warn and higher (warn, error, fatal) to another file eg for support.
From your description it sounds like you want to restrict output to only debug level, and exclude info, warn, error and fatal-level messages. This is possible but not as straight-forward as the use case above.
You probably need to use filters:
http://logging.apache.org/log4j/2.x/manual/filters.html
Take a look at the filter documentation, I'll see if I can dig up an example.

A beginner's log4J question

i configured log4j for basic purpose usin the conversion pattern :-
log4j.appender.rollingFile.layout.ConversionPattern=%p %t %c - %m%n
But now i want to log the class name from which the error came as well as the username(available in session object) as well as the date and time when that event occurs. How do i do this? What changes do i need to make in format string?
Thanks in advance :)

Take a look at the PatternLayout docs for most of what you want.
The headache you face is getting the user name from the session (Log4j can't do this automatically). I would perhaps investigate NDCs or MDCs, and populate these from the session (perhaps in a servlet filter?). They are per-thread, so assuming your user has the same scope then this may help.

To get the class name out you can use %l, but you'll take a bit of performance hit.
To get the username out, you'll need to use a mapped or nested diagnostic context and then specificy %X or %x respectively in the pattern string.
Check the PatternLayout javdocs.