I have 3 question about logging inside srping
First:
spring documentation:
By default, If you use the ‘Starter POMs’, Logback will be used for
logging. Appropriate Logback routing is also included to ensure that
dependent libraries that use Java Util Logging, Commons Logging, Log4J
or SLF4J will all work correctly.
I don't understand that if a third-party library uses a different logger, what problem will be created in the program? If that library uses another logger, that logger is located as a dependency in its jar file, and when the library is added that logger is also added and there is no problem.
second:
I saw in a tutorial that it says that trace and debug are disabled by default in spring because it causes performance problems. I understand why trace is a problem because it must report everything that happens in the program. But why does debug cause performance problems? When I did: debug=true, it didn't take me that much time. So what's the problem?
Third:
In this tutorial, it says that logback does not have a FATAL level. Why not? Is it possible that the spring boot program does not have some of the required settings, but it can still start without the need for FATAL?
Different logging implementations require different configurations. Log4j uses XML and Java-Util-Log (JUL) uses properties. Also the xml sematics differs.
So you do not like to configure all logging implementations individually. You like to configure one logging configuration to rule them all, one source-of-truth for logging-config. This have nothing to do with the main intend of the software you are running. Later logging-frameworks generalize older logging-frameworks, so you need the latest logging-framework to rule them all.
Let me rephrase first: Why do we differ between debug and trace? Debug (or de-bug) is a special condition that let you inspect a bug for debugging purposes. Debug may show clients real-world firstname and family-name, in order to output those informations you need code under debugging-circumstances only. To log them may cause legal problems because you are processing/storing personal informations without permission in log-files. In order to de-bug a software you need the debug-log in 90% of all cases. Only in rare cases you need the trace-log. That meens they differ.
Thats a good one. Fatal for me meens the server has hardware problems (burning hard-drive, loose of power supply). This is indicated by errors. Seriously? I have no idea. I may argue that everything that is fatal-worthit should be an error.
I package my applications as jars and wars. After deployment in production, I like to see on the logs if everything is ok, and once it is, I would like to decrease the level of these logs at runtime, without deploying a new jar/war again.
What is the strategies available to do this with logback? The only I found was to put the logback configuration outside the jar/war, so I could change the level anytime without deploying again. The problem with this is that it will create new arguments to my application, I would like to avoid this, I want to keep it simple.
Is there a better way to accomplish the same?
From my experience, there is no cleaner way than providing an external logback.xml.
Other options include using JMX to adjust the logger level or implementing an endpoint that will change the logger configuration programmatically.
There is inherent tension between logging verbosity and performance of Java app in production. If we log very selectively then we might miss evidences for issues in production to debug . If we add too much logging in production , can impact performance.
I was thinking along the line with couple of options :
Log all selective and important things
Have SSDs instead of hard disks in prod
Have logging utility that can "batch" logging statements and flush periodically
Have some utility that will hold logs in memory and then flush eventually.
What are best approaches other than outlined above ? Are there any existing logging tools that can be used for this purpose ?
Try apache log4j with slf4j( you can switch log4j without much changes in your code). Use the configuration xml to provide what to log and which files to log.
Also use rolling file appenders and buffer appenders to handle flushing and batching of logs.
Jetty 9 is used for the embedded server and everything works well. One thing that remains is the logging issue.
Prior to that mvn:jetty-run brings his own logging setup with it and logs to the console. That is good for development. In the production environment we need something more special.
Currently on start-up the SLF4J complains about, that there is no binding available, so we can chose freely.
That is what we want to archive:
We need to log to the console if we are starting in a non-production environment.
In the production environment the logging should be done in a single log-file but on a daily rotation with the naming schema: logs/logname-date.log (e.g. logs/application-20130926.log)
We distinguish between the production and non-production mode using a command line argument '-production'.
Since the jetty server is embedded I would love to have a solution which we can fully configure the logger without the need to manage xml or properties-files taking the logging configuration aspect out of the deployment process.
So what options do we have and how can we do this in the best possible way?
Update: It seems that logback is the way to go. It has support for the logfile rotation and also makes it possible to use a console output. The difficult question remaining is how to do this programatically and without additional files.
You have hundreds of configuration options here.
You will need to know a few things about your application before you can pick an appropriate configuration here.
How are logging events emitted from your code? the jetty server? and all 3rd party libraries?
Then you want to answer, what logging framework do you want to handle output (to disk, and to console) portions of the logging architecture?
This is documented at Jetty:
http://www.eclipse.org/jetty/documentation/current/example-logging-logback-centralized.html
Yes, that documentation isn't for embedded mode, but it is still relevant.
Your required logging jar files:
The basic api jar:
slf4j-api.jar (required, no matter what you choose below)
The log capturing jars:
(pick [0..n] jars here)
log4j-over-slf4j.jar (slf4j handling of log4j events)
jul-to-slf4j.jar (slf4j handling of java.util.logging events)
jcl-over-slf4j.jar (slf4j handling of jakarta commons-logging events)
The log output jars:
Pick only 1 of the following output jars:
slf4j-simple.jar (this is a super simple logging implementation, not suitable for production)
logback-classic.jar (slf4j's own output logging framework)
also requires logback-core.jar
slf4j-log4j12.jar (route slf4j events to log4j for processing)
also requires log4j.jar
do not use if using log4j-over-slf4j.jar from above
slf4j-jdk14.jar (route slf4j events to java.util.logging for processing)
do not use if using jul-to-slf4j.jar from above
slf4j-nop.jar (route slf4j events to nowhere, silently discard them)
slf4j-jcl.jar (route slf4j events to jakarta commons-logging)
also requires commons-logging.jar and your choice of commons-logging implementation.
do not use if using jcl-over-slf4j.jar from above
Configure it all:
Be sure you read the slf4j manual on each of these jars, as there is sometimes some extra setup details you might need to know about.
For your described situation, the most appropriate output jars would be logback-classic.jar or slf4j-log4j12.jar. As for configuring the output, you would need to rely on the documentation that those libraries provide.
Logback Documentation
Log4j Wiki
So finally here is the complete picture.
After all the logging configuration in a programmatic way is just described here: http://logback.qos.ch/manual/configuration.html#joranDirectly
I use the logback API just as stated by Joakim. Once you learn how to program it programmatically using the JoranConfigurator object everything is quite easy. Play with it and you get the picture.
I managed to accomplished all tasks at hand.
Thanks for the help Joakim. I was missing the JoranConfigurator thingy. Thanks!
Update:
I used a StringReader and embedded the xml configuration file directly in the Logging configuration class. This way I dont have to manage additional files and logging works as expected.
What's the advantage of log4j over set System.out and System.err to output to a log file?
At a high level, the win from Log4j over manual logging is that you can decouple your logging code from what you actually want to log and where and how you want to log it. Details about logging verbosity/filtering, formatting, log location, and even log type (files, network, etc.) are handled declaratively using configuration and extensibly via custom appenders, rather you having to code that flexibility yourself.
This is critically important because it's often hard for developers to predict how logging needs will change once their software is in production. Operations teams managing that software may need less verbose logs, may need mulitple logs, may need to ship those logs to multiple servers, may need to sometimes get really verbose data for troubleshooting, etc. And it's usually impossible for operations teams, if they need to change how logging works, to convince the developer to make big code changes. This often leads to production downtime, friction between operations and development, and wasted time all around.
From the developer's point of view, Log4j insulates you from having to make code changes to support logging, and insulates you from being pestered by people who want logging changes. It enables people managing your code to scratch their own itch rather than bugging you!
Also, since Log4j is the de-facto standard for Java logging, there are lots of tools available which can do cool things with Log4j-- furthermore preventing you and your operations teams from re-inventing the wheel.
My favorite feature is the ability to easily write appenders send data to non-file sources, like SYSLOG, Splunk, etc. which makes it easy to your app's custom logging into operations management tools your IT department is already using.
Actually, you should look into the slf4j facade these days, as it allows you to use {}-placeholders for the most concise statements. You can then use the appropriate logging framework behind slf4j to handle the actual treatment of your log statements. This could be log4j or the slf4j-simple which just prints out all of INFO, WARN and ERROR, and discards the rest.
The crucial observation you need to make is that the WRITING of log statements is done when the code is written, and the DECISION of what is needed is done when the code is deployed, which may be years after the code was written and tested. System.out.println requires you to physically change your code to get rid of them, which is unacceptable in a rigid write-test-deploy cycle. IF the code changes, it must be retested. With slf4j you just enable those you want to see.
We have full logging in the test phase, and rather verbose logging in the initial period of a production deployment, after which we go down to information only. This gives us full information in a scenario where debugging a case is very rarely possible.
You might find this article I wrote interesting. The target audience is beginning Java programmers, with my intention of giving them good habits from the start. http://runjva.appspot.com/logging101/index.html
my favorites (not all)
Ability to set parameters of logging in config, without recompiling
Ability to set the way log is written (from text file to SMTP sender)
Ability to filter by severity
Levels, formatting, logging to multiple files... A logging framework (even if it's java.util.logging) is really beneficial if there's a chance anything may go wrong while your code is running.
log4j allows you to log to various resources e.g. event log, email, file system etc while allowing your application to remain decoupled from all of these resources. Furthermore, you get to use a common interface to log to all of the various resources without having to learn or integrate thier corresponding APIs.
Log4j offers the ability to rotate your log files based on size and delete them based on quantity (logrotate), so your servers don't fill up their disks. Personally I think that is one of the more valuable features in Log4j.
Also Log4j is popular and understood by many developers. The last three companies I've worked at have all used Log4j in most projects.
Take a look and you will understand the power of log4j :
log4j.properties I used once for a project :
# ALL < DEBUG < INFO < WARN < ERROR < FATAL < OFF
# No appenders for rootLogger
log4j.rootLogger=OFF
folder=..
prefix=
fileExtension=.log
htmlExtension=${fileExtension}.html
datestamp=yyyy-MM-dd/HH:mm:ss.SSS/zzz
layout=%d{${datestamp}} ms=%-4r [%t] %-5p %l %n%m %n%n
# myLogger logger
log4j.logger.myLogger=ALL, stdout, infoFile, infoHtml, errorFile
# stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=${layout}
# infoFile
log4j.appender.infoFile=org.apache.log4j.FileAppender
log4j.appender.infoFile.File=${folder}/${prefix}_info${fileExtension}
log4j.appender.infoFile.layout=org.apache.log4j.PatternLayout
log4j.appender.infoFile.layout.ConversionPattern=${layout}
# infoHtml
log4j.appender.infoHtml=org.apache.log4j.FileAppender
log4j.appender.infoHtml.File=${folder}/${prefix}_info${htmlExtension}
log4j.appender.infoHtml.layout=org.apache.log4j.HTMLLayout
log4j.appender.infoHtml.layout.Title=Logs
log4j.appender.infoHtml.layout.LocationInfo=true
# errorFile
log4j.appender.errorFile=org.apache.log4j.FileAppender
log4j.appender.errorFile.File=${folder}/${prefix}_error${fileExtension}
log4j.appender.errorFile.layout=org.apache.log4j.PatternLayout
log4j.appender.errorFile.layout.ConversionPattern=${layout}
# APPENDERS SETTINGS
log4j.appender.stdout.Threshold = ALL
log4j.appender.infoFile.Threshold = INFO
log4j.appender.infoHtml.Threshold = INFO
log4j.appender.errorFile.Threshold = WARN.
To change the variables in your java code you can do :
Loading Configuration
Log4j will automatically load the configuration if it is stored in a
file called "log4j.properties" and is present on the classpath under
"" (e.g. WEB-INF/classes/log4j.properties).
I don't like that approach and prefer to load the configuration
explicitly by calling:
PropertyConfigurator.configure( Config.ETC + "/log4j.properties" );
This way I can reload the configuration at any time as long as my
application is still running. I like to add a button to an
administrative jsp, "Reload Log4J".
Dynamic Log File Location
Many people complain that Log4j forces you to hard-code the location
where your logs will be kept. Actually, it is possible to dynamically
choose the log-file location, especially if you use the ${log.dir}
property substitution technique above. Here's how:
String dynamicLog = // log directory somehow chosen...
Properties p = new Properties( Config.ETC + "/log4j.properties" );
p.put( "log.dir", dynamicLog ); // overwrite "log.dir"
PropertyConfigurator.configure( p );
logging (Document historical business events that occur, you can check old logs)
track the application (project flow)
debugging the application (Detailed information what occurs in a method at granular level //data, value and all inside methods)
error handling (information about specific error that occur)