I've been looking into this issue I've been having a bit, but so far haven't found a single one with the same (or even a similar) problem.
The problem is not in 'how to configure the RollingFileAppender'. It is configured, it is actually rolling over. So far so good.
The context is, each day, every 15 minutes, we execute a batch, of which we write the output to two distinct log files:
a general log file, containing all the output
a functional log, containing only that part of the output which is relevant for the analysts.
The configuration is as follows:
log4j.rootLogger = info, rollingFile, functionalLog
log4j.appender.rollingFile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.rollingFile.File= ../logs/general_log.log
log4j.appender.rollingFile.layout=org.apache.log4j.PatternLayout
log4j.appender.rollingFile.layout.ConversionPattern=%p %d %t %c - %m%n
log4j.appender.rollingFile.DatePattern='.'yyyy-MM-dd
log4j.appender.functionalLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.functionalLog.File= ../logs/functional_log.log
log4j.appender.functionalLog.layout=org.apache.log4j.PatternLayout
log4j.appender.functionalLog.layout.ConversionPattern=%m - %d%n
log4j.appender.functionalLog.filter.1=org.apache.log4j.varia.StringMatchFilter
log4j.appender.functionalLog.filter.1.StringToMatch=FUNC-LOG -
log4j.appender.functionalLog.filter.1.AcceptOnMatch=true
log4j.appender.functionalLog.filter.2=org.apache.log4j.varia.DenyAllFilter
This configuration has worked for months. The batch executed every 15 minutes (triggered by a crontab), and each night, two rolled over files:
general_log.log.2020-05-18
function_log.log.2020-05-18
were created, containing all the loggings of that particular day. The logs were complete, we were happy.
A few days ago, we noticed we were losing logs: apparently, even though the first roll over (at midnight) would succeed, the system would roll over for a second time. Each day, the batch execution of 10.30 am would (again) trigger the (current) logs to be rolled over.
This caused the dated file of the day before, containing all the logs of that day, to be overwritten with the logs of the first ten and a half hours of the current day.
Result: the logs were in the log files of the day before, and we only had about ten hours worth of logs a day, instead of 24.
During the investigation, I noticed I forgot the DatePattern on the FunctionalLog config, so I added this:
log4j.appender.functionalLog.DatePattern='.'yyyy-MM-dd
Now, I know it is possible to set the rollover at yearly, monthly, daily, hourly, each minute, ..., but for some reason, there is some execution each day (10.30 am) that makes it roll over again. There is no config in the log4j, nor in the batch, that can explain this.
There have been no changes in neither the deployed code, the log4j configuration, nor in the configuration of the server.
Has anyone encountered an issue like this before? Is there anything in the config we can do to avoid this?
It turned out the problem wasn't as much the crontab or the log4j configuration.
Apparently a problem on the server caused there to be two processes writing to the log files. We still haven't figured out what might have caused it, but killing the second process solved our issue.
Related
Recently I came across this situation where I have two different types of logs in the server. One is directly using the JDK system time and displaying it another one is using log4j in WLP.
The one which is using log4j is logging time one hour early than the current system time in Austin machines as compared to Asia machines (Assuming time zone issue) (the log without log4j implementation is showing the correct time).
The situation may have occurred due to Day Light Saving. I looked into the log4j2.xml for its tag implementation and found something as follows
current pattern
**<pattern>%d{ISO8601} [%-50.50t] %marker [%p] %F:%L %C.%M() %n......MDC: %X %n......NDC: %x %n......MSG: %-70.70m %n</pattern>**
I tried to change it to the format
**<pattern>%d{yyyy-MM-dd}-%t-%x-%-5p-%-10c:%m%n</pattern>**
to see if it works.
And voila! It started logging the correct time.
[But the situation of restart are different, once completed restarted the machine & once just restarted the WLP, Just after restarting the WLP we found the log time stamp is almost similar to machine time stamp.]
However, I think ISO8601 format is more appropriate to keep.
Have you guys came across such situation or know some solution to it except the one which I tried.
The Time zone information of the server
Clock : UTC
Time Zone : America/Chicago
log4j Version used : log4j-2.x
So, again if you know any solution where daylight saving time can be automatically processed and display the right time with sync. to the current system time or is there any fix available in the latest log4j versions if any, please inform
Thanks
I am using slf4j, implementation of log4j for logging in my java project. Currently I am having 2 appenders, FILE and CONSOLE.
I want to know following 2 things:
Does using multiple appenders (in this case CONSOLE and FILE) causes performance issue in logging?
When somebody would want to use CONSOLE and FILE appenders both?
When writing to CONSOLE and FILE, you are writing to 2 different streams. In a multithreaded system, the performance hit will not be much, but with big volumes it is still apparent.
From the log4J manual
The typical cost of actually logging is about 100 to 300 microseconds.
This includes building the statement and writing it, but the time taken for writing will still be apparent if you are logging heavily.
But you need to ask a more basic question - Why are you logging?
to keep track of what is going on
to find out errors
The CONSOLE is not useful for the first part as the logs are not saved anywhere. If the logging is heavy, and all the logs are sent to the CONSOLE, the amount of logs will make the output on the console unreadable, so purpose 2 is also defeated.
IMO it makes much more sense reading logs from a file using something like less. As a general practice, you log to file and if you must, log only the ERROR messages to console, as a few ERROR messages would be an indicator of something going wrong, whereas hundreds of log lines on console is just junk, as you cannot make any sense of it when the console is refreshing so rapidly.
TL-DR
The cost might not be much, but why incur an added cost when you are getting no added advantage?
Read these links on log 4j performance.
log4j-performance
log4j-decreased application performance
log4j appenders
I challenge you to notice any performance change.
For instance you might want a daemon application to log both in the console and in a file. It does not seem to be such an uncommon behavior.
So I have been trying to run a Hadoop MapReduce job for a while, and after it successfully started running (all errors sorted out), I wanted to check the log files for the stdout which is captured in the log files, but somehow I see that the log files aren't being generated every time. (some times it comes, other time it doesn't)
I was using an output directory (/user/hduser/output_dir), and deleting the contents and using that again (to avoid keeping track of so many output directories) but the log files indicate the time when the last change was made to it, and that does not match the last time I ran the job.
Also, the log files in /user/hduser/output_dir does not match with $HADOOP_HOME/logs/userlogs
Is it a known problem, and is there any solution to this? I did not find the answer to it anywhere.
Thanks for the help.!
EDIT - So we found out that the log files are being written only after certain intervals of time, so if a job is run twice within this time then new log files aren't written for that. Why would this be so, and how can I override it using some configuration changes, if it's possible?
I have a "strange" problem. I use log4j in a Java application.
On my PC the times that appear in log4j generated logs are fine, corresponding to the local time of my machine.
The problem is that when I deploy the application on a Windows Server 2003, the times that appear in the log files are going haywire. The date is fine, but the hours and minutes not at all. They don't match at all the server's local time. And I doesn't seem to be a time zone issue since the difference between the local time and log time seems to be 4h 30 minutes. I know, there are some timezones with 30 minutes, but, still...
Does anybody know how is this possible?
Here you have the conversion pattern for appender R:
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
I put only this part of the log4j.properties because only this one is interesting for my problem.
Still, it looks like a time zone problem. Don't you, for example, run the application on a different user account than you use for checking the date on the server?
Log4j takes the default time zone, so you could try running your application with the additional command line option:
-Duser.timezone=UTC
and see if it anything changes.
Are there any known bugs with the Log4J rolling file appender. I have been using log4j happily for a number of years but was not aware of this. A colleague of mine is suggesting that there are known issues ( and i found one a Bugzilla entry on this) where under heavy load,the rolling file appender (we use the time-based one) might not perform correctly when the rollover occurs # midnight.
Bugzilla entry - https://issues.apache.org/bugzilla/show_bug.cgi?id=44932
Appreciate inputs and pointers on how others overcome this.
Thanks,
Manglu
I have not encountered this issue myself, and from the bug report, I would suspect that it is very uncommon. Th Log4j RollingFileAppender has always worked in a predictable and reliable fashion for the apps I have developed and maintained.
This particular bug, If I understand correctly, would only happen if there are multiple instances of Log4j, like if you had multiple instances of the same app running simultaneously, writing to the same log file. Then, when it is rollover time, one instance cannot get a lock on the file in order to delete it and archive its contents, resulting in the loss of the data that was to be archived.
I cannot speak to any of the other known bugs your colleague mentioned unless you would like to cite them specifically. In general, I believe Log4j is reliable for production apps.
#kg, this happens to me too. This exact situation. 2 instances of the same program.
I updated it to the newer rolling.RollingFileAppender instead of using DailyFileRoller( whatever it was called ).
I run two instances simultenously via crontab. The instances output as many messages as they can until 5 seconds is reached. They measure the time for 1 second by using System.currentTimeMillis, and append to a counter to estimate a 5 second timeperiod for the loop. So there's minimum overhead in this test. Each output log message contains an incrementing number, and the message contains identifiers set from commandline to be able to separate them.
From putting the log message order together, one of the processes succeeds in writing from the start to end of the sequence, the other one loses the first entries of its output (from 0 onward).
This really ought to be fixed...