I'm using https://logback.qos.ch/ for a long running Java program.
Is there a way to configure some appender to log only the first N lines of a program?
For example, my program logs important information at startup but if I use a rolling file appender, the logs at startup are eventually deleted.
I found how to do this in log4j as follows:
<appender name="StartupAppender" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="${samza.log.dir}/${samza.container.name}-startup.log" />
<param name="MaxFileSize" value="256MB" />
<param name="MaxBackupIndex" value="1" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} [%p] %m%n" />
</layout>
</appender>
<logger name="STARTUP_LOGGER" additivity="false">
<level value="info" />
<appender-ref ref="StartupAppender"/>
</logger>
How do I do this in Logback?
There is no canned, pre-existing Logback appender which will "Log only first N lines with Logback" I'd suggest that you use Logback's existing implementations of rollingPolicy and triggeringPolicy to create a Logback configuration which matches the configuration you had working in Log4J.
The Logback equivalent of the StartupAppender in your question is:
<appender name="StartupAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${samza.log.dir}/${samza.container.name}-startup.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${samza.log.dir}/${samza.container.name}-startup.log.%i.log</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>1</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>256MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} [%p] %m%n</pattern>
</encoder>
</appender>
<logger name="STARTUP_LOGGER" additivity="false">
<level value="info" />
<appender-ref ref="StartupAppender"/>
</logger>
This will behave the same as your Log4J StartupAppender but just like that appender it is not guaranteed to retain the "first N lines of a program" since it will rollover on reaching 256MB. However, since you are only associating this appender with the logger: STARTUP_LOGGER it seems likely that 256MB would be more than enough space to retain a lengthy history of 'startup logs'.
Related
I am trying to create logger policy in a spring based project.
The issue I am facing is related to rolling policy. the logfile.log is created and is working fine but the rolling file rollingfile.log.%d{yyyy-MM-dd}.log is not created.
Given below is my logback.xml.
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true" scan="true" scanPeriod="30 seconds">
<property name="LOG_PATH" value="D:/coinLogs" />
<property name="LOG_ARCHIVE" value="${LOG_PATH}/archive" />
<appender name="Console-Appender" class="ch.qos.logback.core.ConsoleAppender">
<layout>
<pattern>[%d{yyyy-MM-dd HH:mm:ss}] - [%X{requestId}] - %p %c -- %m%n</pattern>
</layout>
</appender>
<appender name="File-Appender" class="ch.qos.logback.core.FileAppender">
<file>${LOG_PATH}/logfile.log</file>
<encoder>
<pattern>[%d{yyyy-MM-dd HH:mm:ss}] - [%X{requestId}] - %p %c -- %m%n
</pattern>
<outputPatternAsHeader>true</outputPatternAsHeader>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${LOG_ARCHIVE}/rollingfile.log.%d{yyyy-MM-dd}.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<logger name="coinPay.logbackxml" level="info" additivity="false">
<appender-ref ref="Console-Appender" />
<appender-ref ref="File-Appender" />
</logger>
<!-- To remove extra hibernate logs -->
<logger name="org.hibernate">
<level value="info" />
</logger>
<root>
<appender-ref ref="Console-Appender" />
<appender-ref ref="File-Appender" />
</root>
</configuration>
any help will be appreciated. Thank you in advance :D
You need to specify ch.qos.logback.core.rolling.RollingFileAppender as class for your appender. A FileAppender can not roll.
I believe you're missing the %i parameter, which should be replaced by the rolling file index (i.e. 'mylog.2017-03-13.0.txt', 'mylog.2017-03-13.1.txt', etc.)
In addition, like Uwe Allner, I'm using RollingFileAppender instead of FileAppender.
(I also suggest adding a totalSizeCap to the configuration, in order to control to total size)
Here's my entire appender configuration:
<appender name="SQL_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_HOME}/mylog.txt</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level- %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<maxHistory>3</maxHistory>
<totalSizeCap>5GB</totalSizeCap>
<fileNamePattern>${LOG_HOME}/archived/mylog.%d{yyyy-MM-dd}.%i.txt.zip</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>20MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
I have a service class that import and parse files from an ftp server.
This task can be executed from a Spring controller when the user choose a specific action.
The same service is used by a scheduled task (TimerTask) that everyday at a specific time do the same.
I would separate logging for user / scheduled execution into different files:
logs/scheduledImport.log
logs/userImport.log
Is it possible with Log4j?
Of course. First you must define 2 different file appenders.
<appender name="default.file" class="org.apache.log4j.FileAppender">
<param name="file" value="/log/userImport.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
<appender name="another.file" class="org.apache.log4j.FileAppender">
<param name="file" value="/log/scheduledImport.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{ISO8601} %-5p [%c{1}] - %m%n" />
</layout>
</appender>
Then you define your loggers:
<logger name="com.foo.UserService" additivity="false">
<level value="debug" />
<appender-ref ref="default.file" />
</logger>
<logger name="com.foo.ScheduledService" additivity="false">
<level value="debug" />
<appender-ref ref="another.file" />
</logger>
You can use different loggger instances. In this case you can apply different settings (output file) for each logger.
I want to redirect log to specific log file, by classname. The intention is to have a group of classes log output to a specific log file.
Right now the all output comes to console, and on startup it throws and error saying log4j:ERROR Could not retrieve category [com.mycomp]. Reported error follows..
What am I doing wrong here?
In my java class :-
private static final Logger log = Logger.getLogger(SpringBootLog4jApplication.class.getName());
My log4j.xml :-
<?xml version="1.0" encoding="UTF-8" ?>
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/' debug="false">
<appender name="LOGFILE" class="org.apache.log4j.RollingFileAppender">
<param name="file" value="/git/services.log"/>
<param name="maxBackupIndex" value="10"/>
<param name="maxFileSize" value="20MB"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %x %p %c{1} :: %m\n"/>
</layout>
</appender>
<appender name="stdout" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<!-- Pattern to output the caller's file name and line number -->
<param name="ConversionPattern" value="%5p [%t] (%F:%L) - %m%n"/>
</layout>
</appender>
<logger name="com.mycomp" class="com.mycomp.SpringBootLog4jApplication" additivity="false">
<level value="INFO"/>
<appender-ref ref="LOGFILE"/>
</logger>
<root>
<level value="INFO"/>
<appender-ref ref="stdout"/>
</root>
I believe that the name for the logger should look like
<logger name="com.mycomp.*"...
as that would resolve to any concrete classes in the package.
I want to set up log4j so that all log meessages produced from classes under package com.foo.bar go to bar.log, and all the log meessages produced from classes under package com.bar.blatz go to blatz.log.
Questions
How do I do this using log4j.xml?
I know its possible using property files, but how do I do it using the XML configuration?
This is based on my answer to a similar question:
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<!-- general application log -->
<appender name="BarLogFile" class="org.apache.log4j.FileAppender">
<param name="File" value="bar.log" />
<param name="Threshold" value="INFO" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p %t [%-40.40c] %x - %m%n"/>
</layout>
</appender>
<!-- additional fooSystem logging -->
<appender name="BlatzLogFile" class="org.apache.log4j.FileAppender">
<param name="File" value="blatz.log" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%-5p %t [%-40.40c] %x - %m%n"/>
</layout>
</appender>
<logger name="com.foo.bar">
<appender-ref ref="BarLogFile"/>
</logger>
<logger name="com.bar.blatz">
<appender-ref ref="BlatzLogFile"/>
</logger>
<root>
<level value="INFO"/>
<!-- no appender, output will be swallowed (I think) -->
</root>
</log4j:configuration>
If you add an appender-ref to the root element, it will also receive com.foo.bar etc messages. You can stop that by specifying 'additivity="false"' on the loggers.
<root>
<level value="INFO"/>
<!-- no appender, output will be swallowed (I think) -->
</root>
We can add appenders here. It will work if the application is using root logger. for example quartz Scheduler API.
I have a problem with the configuration of my logging with log4j. All the log messages are shown as expected in the root.log file, but the stdout, however, does only show a subset of the root.log although I configured it to be the same.
EDIT: Precisely I am missing all messages from a.b.
EDIT 2: I am not absolutely sure, but I think that some messages from a.b. (INFO and DEBUG) get loggedd on stdout, others don't. Might that be a problem related to threads? E.g. some threads doing logging, some not?
The following is my log4j.xml, is there eventually a mistake?
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd" >
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{HH:mm:ss,SSS} [%t] (%C:%L) %-5p - %m%n" />
</layout>
</appender>
<appender name="FILE_ALL" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="${catalina.home}/logs/root.log"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-4r (%F:%L) %-5p %x - %m%n" />
</layout>
</appender>
<logger name="a.b.master">
<level value="DEBUG" />
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE_ALL" />
</logger>
<logger name="a.b.master.orm.support.HibernateSessionFilter">
<level value="INFO" />
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE_ALL" />
</logger>
<logger name="a.b.master.aop.hibernate.OpenSessionInRequestInterceptor">
<level value="info" />
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE_ALL" />
</logger>
<root>
<level value="INFO"/>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE_ALL"/>
</root>
</log4j:configuration>
I am using Commons Logging as implementation.
Thanks for the help!
Your configuration is very similar to mine where my logs are dumped into the console and to some remote host. Try this to see if it works for you:-
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
...
</appender>
<appender name="FILE_ALL" class="org.apache.log4j.RollingFileAppender">
...
</appender>
<logger name="a.b.master">
<level value="debug" />
</logger>
<logger name="a.b.master.orm.support.HibernateSessionFilter">
<level value="info" />
</logger>
<logger name="a.b.master.aop.hibernate.OpenSessionInRequestInterceptor">
<level value="info" />
</logger>
<root>
<priority value="info"/>
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE_ALL"/>
</root>
</log4j:configuration>
I don't think you need to define appender-ref in every a.b logger since it should pick up the appender from root. Also, I use priority instead of logger for root. Otherwise, your configuration looks identical to mine and I'm able to get mine working properly. If this doesn't work for you for some reason, I would suggest you to remove your custom a.b loggers from the configuration to see if you are able to get the a.b messages in your console.
Also, just FYI, I'm sure you already know this, but if your root is set to use "info", then there's really no need to set your custom a.b logger to "info" too. I usually leave my root to "warn" and have my custom loggers to override that with either "debug" or "info".
I found the solution even if not the problem directly. I added a parameter to my ConsoleAppender that everything is printed to the stderr. This now works. I guess its a problem of eclipse or log4j because if i am starting the app from the terminal (console) I can see all log messages properly.
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.err" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d [%t] %-4r (%F:%L) %-5p %x - %m%n" />
</layout>
</appender>