I have been working on a project that logs using what are essentially just println statements with a prefixed string tag. I have been looking into implementing support for an actual logging library such as Logback the past few days and had some questions relating to best-practices about logging in general. I know a lot of what I'm doing is probably stupid, but I want to change :)
When I'm extending the code and adding new features, such as testing a new codec, I have been using liberal logging to ensure the code behaves as expected (instead of actual unit tests), and then using constant booleans at the top to disable that logging when the codec is finished (in case it's needed again or a bug is found, I can flip the boolean while testing). I don't know if the granularity that debug level provides would be enough and would prefer some way to define levels differently for different features. Leaving these enabled by default would really bloat the console and probably effect performance -- is this what filters are usually used for?
I've also found myself in more than one case prepending spaces to my messages so that I can better follow the flow of the code. I've found this to be really helpful. In a way, the tabbed messages are like a debug-debug level.
Doing something
Reading a file
header of file: ...
body of file: ...
Back at main
What are good practices for logging? Can someone refer me a good resource that I can dig into or explain if what I'm doing is stupid and why it's stupid? What are some alternatives? An open source project as an example would be extremely helpful. Thanks, I appreciate any guidance.
Some advice:
Never substitute unit tests for logging only.
Regarding logging you should log whatever makes you find bugs quicker.
Log libraries support async logging which will not affect the performance of your application (log4j2 async logging). Logback supports async too.
Do not use booleans inside your code to decide to log or not to log. Use the logging levels (TRACE, DEBUG, INFO, WARN, ERROR) and set them accordingly. Usually in PROD environment you will use a WARN level and in DEV you can set it on debug.
Logging on different levels depending on the package is quite useful (you can use appenders to customize this and other stuff).
In resume, the important is to read the documentation of whatever library you are using
How to implement log4j such that some desired loggers are not shown at PRODUCTION environment, but will show at test and acceptance environment.
Is it possible to do by using log4j only?
You can use maven profiles . use different log4j.xml config file for every environment
You Can use log4j for Production. But make sure that there is minimum usage of log in production. As less info and fatal can lead to less logger in production.
Lesser the logging faster the process, More the logging ,more verbose will be the server.
use warn precisely and only when there is any transaction happening with Database.
Hope it could help you!!!
You can work with the log4j defaults logs Levels, for example use minimal level DEBUG for test or acceptance environment and INFO for production.
Besides, you can implement your own custom log levels https://logging.apache.org/log4j/2.x/manual/customloglevels.html
What is performance wise advantage of using log4j over System.out.Println?
FYI:I know log4j has multiple appenders,debug logging and other features which System.Out.println doesn't have and is applicable at class level also and is used in larger applications.
But if I have a small application, say a file will log4j will provide better performance than System.Out.println. How internally log4j works?
Log4j isn't entitled to be more performant. It was created for having more abilities to decrease log amounts and specify the log output. Imagine a Tomcat server which logs amounts of hibernate database accesses. E.g. with the log level you can stop straining the server through this info logs. But this is not a "native" performance advantage since you can simulate this with flag checking before sysos.
The only case I can think of in which you could perceive a performance advantage using log4j over System.out.println would be for example if doing this:
logger.debug("Logging something");
Instead of doing this:
System.out.println("Logging something");
And you then configure the logging level in such a way that it does not require logging debug messages.
In that scenario, the first line will not actually log, but the second one will still write to the System.out.
In all other cases (I mean, in all cases where logging is actually done) I don't think it will be faster to use log4j performance wise.
Usually it's used for all the reasons you state, and because the performance is not that different (framework overhead is minimal compared to the time it actually takes to write things on files).
The prudent mode in logback serializes IO operations between all JVMs writing to the same file, potentially running on different hosts. In other logging frameworks, logging to a central TCP (or JMS) appender seems to be the only solution if output from many loggers should go to the same file.
As I am using a Delphi library which is based on log4j and also can not log to the same file from different instances of the same applications (on a terminal server), it would be interesting to know how this feature is implemented. - p.s. I'll check the logback source code and come back to answer my question if nobody is faster :)
It's implemented with a simple FileLock. You can check in the source of FileAppender.
What's the advantage of log4j over set System.out and System.err to output to a log file?
At a high level, the win from Log4j over manual logging is that you can decouple your logging code from what you actually want to log and where and how you want to log it. Details about logging verbosity/filtering, formatting, log location, and even log type (files, network, etc.) are handled declaratively using configuration and extensibly via custom appenders, rather you having to code that flexibility yourself.
This is critically important because it's often hard for developers to predict how logging needs will change once their software is in production. Operations teams managing that software may need less verbose logs, may need mulitple logs, may need to ship those logs to multiple servers, may need to sometimes get really verbose data for troubleshooting, etc. And it's usually impossible for operations teams, if they need to change how logging works, to convince the developer to make big code changes. This often leads to production downtime, friction between operations and development, and wasted time all around.
From the developer's point of view, Log4j insulates you from having to make code changes to support logging, and insulates you from being pestered by people who want logging changes. It enables people managing your code to scratch their own itch rather than bugging you!
Also, since Log4j is the de-facto standard for Java logging, there are lots of tools available which can do cool things with Log4j-- furthermore preventing you and your operations teams from re-inventing the wheel.
My favorite feature is the ability to easily write appenders send data to non-file sources, like SYSLOG, Splunk, etc. which makes it easy to your app's custom logging into operations management tools your IT department is already using.
Actually, you should look into the slf4j facade these days, as it allows you to use {}-placeholders for the most concise statements. You can then use the appropriate logging framework behind slf4j to handle the actual treatment of your log statements. This could be log4j or the slf4j-simple which just prints out all of INFO, WARN and ERROR, and discards the rest.
The crucial observation you need to make is that the WRITING of log statements is done when the code is written, and the DECISION of what is needed is done when the code is deployed, which may be years after the code was written and tested. System.out.println requires you to physically change your code to get rid of them, which is unacceptable in a rigid write-test-deploy cycle. IF the code changes, it must be retested. With slf4j you just enable those you want to see.
We have full logging in the test phase, and rather verbose logging in the initial period of a production deployment, after which we go down to information only. This gives us full information in a scenario where debugging a case is very rarely possible.
You might find this article I wrote interesting. The target audience is beginning Java programmers, with my intention of giving them good habits from the start. http://runjva.appspot.com/logging101/index.html
my favorites (not all)
Ability to set parameters of logging in config, without recompiling
Ability to set the way log is written (from text file to SMTP sender)
Ability to filter by severity
Levels, formatting, logging to multiple files... A logging framework (even if it's java.util.logging) is really beneficial if there's a chance anything may go wrong while your code is running.
log4j allows you to log to various resources e.g. event log, email, file system etc while allowing your application to remain decoupled from all of these resources. Furthermore, you get to use a common interface to log to all of the various resources without having to learn or integrate thier corresponding APIs.
Log4j offers the ability to rotate your log files based on size and delete them based on quantity (logrotate), so your servers don't fill up their disks. Personally I think that is one of the more valuable features in Log4j.
Also Log4j is popular and understood by many developers. The last three companies I've worked at have all used Log4j in most projects.
Take a look and you will understand the power of log4j :
log4j.properties I used once for a project :
# ALL < DEBUG < INFO < WARN < ERROR < FATAL < OFF
# No appenders for rootLogger
log4j.rootLogger=OFF
folder=..
prefix=
fileExtension=.log
htmlExtension=${fileExtension}.html
datestamp=yyyy-MM-dd/HH:mm:ss.SSS/zzz
layout=%d{${datestamp}} ms=%-4r [%t] %-5p %l %n%m %n%n
# myLogger logger
log4j.logger.myLogger=ALL, stdout, infoFile, infoHtml, errorFile
# stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=${layout}
# infoFile
log4j.appender.infoFile=org.apache.log4j.FileAppender
log4j.appender.infoFile.File=${folder}/${prefix}_info${fileExtension}
log4j.appender.infoFile.layout=org.apache.log4j.PatternLayout
log4j.appender.infoFile.layout.ConversionPattern=${layout}
# infoHtml
log4j.appender.infoHtml=org.apache.log4j.FileAppender
log4j.appender.infoHtml.File=${folder}/${prefix}_info${htmlExtension}
log4j.appender.infoHtml.layout=org.apache.log4j.HTMLLayout
log4j.appender.infoHtml.layout.Title=Logs
log4j.appender.infoHtml.layout.LocationInfo=true
# errorFile
log4j.appender.errorFile=org.apache.log4j.FileAppender
log4j.appender.errorFile.File=${folder}/${prefix}_error${fileExtension}
log4j.appender.errorFile.layout=org.apache.log4j.PatternLayout
log4j.appender.errorFile.layout.ConversionPattern=${layout}
# APPENDERS SETTINGS
log4j.appender.stdout.Threshold = ALL
log4j.appender.infoFile.Threshold = INFO
log4j.appender.infoHtml.Threshold = INFO
log4j.appender.errorFile.Threshold = WARN.
To change the variables in your java code you can do :
Loading Configuration
Log4j will automatically load the configuration if it is stored in a
file called "log4j.properties" and is present on the classpath under
"" (e.g. WEB-INF/classes/log4j.properties).
I don't like that approach and prefer to load the configuration
explicitly by calling:
PropertyConfigurator.configure( Config.ETC + "/log4j.properties" );
This way I can reload the configuration at any time as long as my
application is still running. I like to add a button to an
administrative jsp, "Reload Log4J".
Dynamic Log File Location
Many people complain that Log4j forces you to hard-code the location
where your logs will be kept. Actually, it is possible to dynamically
choose the log-file location, especially if you use the ${log.dir}
property substitution technique above. Here's how:
String dynamicLog = // log directory somehow chosen...
Properties p = new Properties( Config.ETC + "/log4j.properties" );
p.put( "log.dir", dynamicLog ); // overwrite "log.dir"
PropertyConfigurator.configure( p );
logging (Document historical business events that occur, you can check old logs)
track the application (project flow)
debugging the application (Detailed information what occurs in a method at granular level //data, value and all inside methods)
error handling (information about specific error that occur)