We Keep Coding

Log4j2 java properties based configuration not deleting old files

The below log4j2 configuration is failing to delete my old files.
# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf-%d{yyyy-MM-dd-HH-mm}-%i.log.zip
# uncomment to not force a disk flush
# log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
log4j2.appender.rolling.policies.time.interval = 1
log4j2.appender.rolling.policies.time.modulate = true
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 5MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.fileIndex = nomax
# uncomment to set max number of files to create per day
# log4j2.appender.rolling.strategy.max = 10
log4j2.appender.rolling.strategy.action.type = Delete
log4j2.appender.rolling.strategy.action.basePath = ${karaf.data}/log
log4j2.appender.rolling.strategy.action.maxdepth = 1
log4j2.appender.rolling.strategy.action.condition.type = IfFileName
log4j2.appender.rolling.strategy.action.condition.glob = karaf-*.log.zip
log4j2.appender.rolling.strategy.action.condition.nested_condition.type = IfLastModified
log4j2.appender.rolling.strategy.action.condition.nested_condition.age = 5m
My log files are being generated correctly as setup by the Time Based and Size Based policies but I cannot figure out why the files aren't being deleted. Everything seems to be correct and I am getting no errors in my logs.
I have no leading or trailing spaces.
I've wasted so much time trying to debug this. The official log4j2 documentation is woeful as it doesn't give java properties based config examples.
Any ideas on how I can fix this?
Thanks

Got it working by changing:
log4j2.appender.rolling.strategy.action.condition.glob = karaf-*.log.zip
to
log4j2.appender.rolling.strategy.action.condition.glob = *.zip
Looks like the old regex pattern isn't being interpreted properly which is extremely strange since it is definitely correct.

FileHandler generate extra logs files when original log is locked

I need generate one log file by each application installed and running on websphere application server 9.
I use JUL for generate log's file. My solution was create a especific Class thas inherits from FileHandler and set logs properties by a config file.
This is my code:
//Read config file
LogManager.getLogManager().readConfiguration(LoggerJUL.class.getResourceAsStream("/Logger.properties"));
//Add handler to logger
Logger.getLogger(clazz)).addHandler(new PersonalFileHandler());
PersonalFileHandler extends FileHandler, and properties are set by configure method on FileHandler class on runtime.
In this way i achieve make one log file by application running over Websphere, without overwriting the destination of the server log.
Although I achieve part of the objective, extra files are generated if the original log file is locked, same like this: testLogs.log.0, testLogs.log.1, testLogs.log.0.1, etc.
I read many suggestions and solutions, but i can't stop this isue.
Any suggestions ?
handlers = com.mucam.xxxx.PersonalFileHandler
# Set the default formatter to be the simple formatter
com.mucam.xxxx.PersonalFileHandler.formatter = java.util.logging.SimpleFormatter
# Write the log files to some file pattern
com.mucam.xxxx.PersonalFileHandler.pattern = C:/Users/pmendez/Documents/Log/testLogs.log
# Limit log file size to 5 Kb
com.mucam.xxxx.PersonalFileHandler.limit = 5000
# Keep 10 log files
com.mucam.xxxx.PersonalFileHandler.count = 10
#Customize the SimpleFormatter output format
java.util.logging.SimpleFormatter.format = %d{ISO8601} [%t] %-5p %c %x - %m%n
#Append to existing file
com.mucam.xxxx.PersonalFileHandler.append = true

Although I achieve part of the objective, extra files are generated if the original log file is locked, same like this: testLogs.log.0, testLogs.log.1, testLogs.log.0.1, etc. I read many suggestions and solutions, but i can't stop this isue. Any suggestions ?
Since your count is set to 10 you need to specify the %g pattern to log the generation.
com.mucam.xxxx.PersonalFileHandler.pattern = C:/Users/pmendez/Documents/Log/testLogs%g.log
The pattern is absolute path so if you create multiple filehandlers it will resolve conflicts by appending unique number to the end. This is specified by the %u pattern. So if you want to move where the integer is placed in the file name you specify the %u token in the pattern.
This also means that you are creating multiple instances of your custom file handler and they are not being closed. If you want to control the number of files you either need to control the number of PersonalFileHandler you create and or share a reference to singleton PersonalFileHandler. Otherwise you need to make sure that if you explicitly create a PersonalFileHandler that you are explicit closing that PersonalFileHandler before you create a second new PersonalFileHandler.
Loggers are subject to garbage collection. The line:
Logger.getLogger(clazz).addHandler(new PersonalFileHandler());
Is subject to garbage collection unless the code elsewhere as already pinned that logger in memory. This can cause log files to be created, locked, and empty.

Log4j2 RollingFile: Use todays date in current logfile (refreshing fileName)

I had a webservice using log4j, and upgraded it to use log4j2 so I could take advantage of the way log4j2 can delete old log dated log files for you while log4j can't.
The logs are stored in the pattern /my_log_root/yyyyMMdd/myservice.log. The relevant log4j.properties config looked like this:
log4j.appender.RollingAppender=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.RollingAppender.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.RollingAppender.RollingPolicy.FileNamePattern=/my_log_root/%d{yyyyMMdd}/myservice.log
This worked well, except old log files were never deleted. Now in log4j2 the properties file looks like this:
property.basePath = /my_log_root/
appender.rolling.type = RollingFile
appender.rolling.fileName= ${basePath}/${date:yyyyMMdd}/myservice.log
appender.rolling.filePattern= ${basePath}/%d{yyyyMMdd}/myservice.log
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.delete.type = Delete
appender.rolling.strategy.delete.basePath = ${basePath}
appender.rolling.strategy.delete.maxDepth = 3
appender.rolling.strategy.delete.ifLastModified.type = IfLastModified
appender.rolling.strategy.delete.ifLastModified.age = 30d
With this properties file, the old log files are deleted (the empty directories remain, but that's not too bad).
The only problem is with the current days log. It appears as if log4j2 is never refreshing the fileName once it's calculated. For example, when I relaunched the service on 2019-06-21, it wrote the file correctly to /my_log_root/20190621/myservice.log. However, now a few days later, it's still writing the current log to /my_log_root/20190621/myservice.log. At the end of the day it will rotate the log to the appropriate dated directory, and then start writing the new log to the 21st.
Is it possible to make log4j2 refresh it's current logs location each day or does it not support this?

Understanding of Max Backup Index in log4j framework

I will introduce to you a simple source example:
#Level
log4j.rootLogger = DEBUG, f
#Appender
log4j.appender.f = org.apache.log4j.RollingFileAppender
log4j.appender.f.File = C:\\Users\\myname\\Desktop\\Test.txt
#File size
log4j.appender.f.MaxFileSize = 100KB
log4j.appender.f.MaxBackupIndex = 1
I understand the working of source and see that the output result will be "logger messages" written in file Test.txt. And when I have reached the max file size of 100KB it will be created new file with name Test.txt.1
My simple issue is can I generate new file to be with name Test1.txt no Test.txt.1
Best regards,
D.Balamjiev

You may want to use a custom FileNamePattern, using %i which is the index of the file :
#Appender
log4j.appender.f = org.apache.log4j.RollingFileAppender
log4j.appender.f.File = C:\\Users\\myname\\Desktop\\Test.txt
log4j.appender.f.rollingPolicy.FileNamePattern=C:\\Users\\myname\\Desktop\\Test%i.txt

That parameter is defining how many files will be kept after deleting from rollback policy:
the official doc states:
maxBackupIndex: Maximum number of backup files to keep.
in your case that will happen every time the loger file reaches the 100KB you defined...

Thanks to Berger for the idea.
Here is my full working source
log4j.rootLogger = DEBUG, Roller
log4j.appender.Roller=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.Roller.RollingPolicy=org.apache.log4j.rolling.FixedWindowRollingPolicy
log4j.appender.Roller.RollingPolicy.maxIndex=5
log4j.appender.Roller.TriggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.Roller.TriggeringPolicy.MaxFileSize= 50000
log4j.appender.Roller.RollingPolicy.ActiveFileName=D:\\Test\\Fixed.txt
log4j.appender.Roller.layout = org.apache.log4j.PatternLayout
log4j.appender.Roller.layout.ConversionPattern = Date: %d{dd-MM-yyyy} Time: %d{HH:mm:ss} [Message is: %m] Other/%r [%t] %-5p %c %x/%n

File Handler doesn't start working until i have a file created

I have a loggers.config file that specifies a custom file handler to format log messages to a file. The handler wont work unless a create a default file with the specified file pattern. It then creates a new file with a 0 appended to it as specified by the pattern and starts logging to that. How can I get it to log to a file without specifying a default file first.
Here is the logging file:
handlers=com.daniel.logging.MyFileHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility specific level
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level= INFO
com.daniel.logging.MyFileHandler.level=INFO
# Naming style for the output file:
com.daniel.logging.MyFileHandler.pattern=daniel%g.log
# Limiting size of output file in bytes -defaulted to 1MB:
com.daniel.logging.MyFileHandler.limit=1MB
# Number of output files to cycle through, by appending an
# integer to the base file name:
com.daniel.logging.MyFileHandler.count=10
# Style of output (Simple or XML):
com.daniel.logging.MyFileHandler.formatter=com.daniel.logging.MyLogFormatter
# Limit the message that are printed on the console to INFO and above.
java.util.logging.ConsoleHandler.level = INFO
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
In the installer (this is an application logging properties file there was a file created there is a line specifying the logger.config file as logger.properties. A couple of lines later a file was created and the command CMD [#] >> default_file 2>&1. I am not sure what that line does (I'm quite new to bash)

Check that the user account the program is executing under has write permissions to the file path.
com.daniel.logging.MyFileHandler.limit=1MB
That should be changed to:
com.daniel.logging.MyFileHandler.limit=1048576
Your file pattern is using a relative path. You should use a absolute path or use the patterns for the home or temp directory. The FileHandler will fail to create a new log file if the directory path doesn't exist. The directories have to exist before the FileHandler will create the log file.
If you want to SimpleDateFormat patterns in the file pattern you can do something like:
public final class MyFileHandler extends FileHandler {
public MyFileHandler() throws IOException {
super(toPattern());
}
private static String toPattern() throws IOException {
String defaultPattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ";
String p = MyFileHandler.class.getName();
String v = LogManager.getLogManager().getProperty(p + ".pattern");
if (v == null) {
v = defaultPattern;
}
try {
return new SimpleDateFormat(v).format(new Date());
} catch (RuntimeException re) {
throw new IOException(v, re);
}
}
}
If you need rotation support you can reference PackageNameFileHandler as a starting point.