In the log4j.properties file I've set the Level to ERROR. For certain users I need to set Level to DEBUG. I was able to change the logging level at run time, but this will be enabled for all the users accessing the application at the same time.
Is there any other method by which we can enable logging for selected users? Any help will be greatly appreciated.
I presume you have a web application or similar, and multiple identifiable users accessing it simultaneously ?
You can't easily alter configuration per user in Log4j. However I would consider the following (this assumes a web server or similar, with each user request being on a separate thread):
identify the user making a server call as soon as they make that call. Store that user data in a MDC
Implement a custom Log4J appender. For each incoming call, you can inspect the user stored in the MDC, and adjust the severity/logging as required.
That's a little bit of work (given the above assumptions) but should work. Obviously it's not valid if the assumptions I've made about your architecture are incorrect.
This problem is solved in logback (log4j's successor) with TurboFilters. See the example entitled "MDCFilter and MarkerFilter configuration". If you need further help, contact the logback-user mailing list.
Yes, but.
You could make your loggers named with the class + user involved, rather than just the class, and create the logger on the method (or if you want to get fancy, cache loggers in some kind of pool keyed by user), and then configure logging appropriately.
It is very messy and intrusive to the code, but since the user is a runtime property, I don't see how (short of AspectJ or its cousins) how you avoid that kind of mess.
Another option is to specially format your log messages and include the user name in the log message, and then parse the logs afterwards. This would enable debugging for everyone (which may be a performance issue, obviously) but if the concern is more about isolating a users logging rather than limiting the amount of debug calls, that may be a solution.
Related
I'm so worried about people logging confidential information to server logs.
I have seen server logs in production. Some developers are accidentally logging security related
information like password, clientId, clientSecret etc.
Is there any way, like Eclipse plugin or any tool, to warn developers while writing their code?
`ex : log.info("usernam = " + username + "password = " + password) ;` //
Warn that confidential info is getting logged.
I have done some research... I have seen tools like sonarLint and FindBug
but those plugins are unable to solve my problem.
SonarLint offers the rule S2068: Credentials should not be hard-coded, which targets the use of hard-coded credentials, and it seems close to what you are trying to achieve, though it may be not enough for your needs.
As stated in other answers, however, identifying such security holes can be ultimately hard and strong code reviews is certainly a good move to reduce the risks.
Now, if you really fear about usages of loggers, already knows potential issues, and what data could leak, I would suggest to write your own Java Custom Rule for SonarQube.
Custom rules are supported by SonarLint and can be applied at enterprise level once the Custom Plugin containing it is deployed on a SonarQube server. This solution would allow you to explicitly define what you want to target, and fine-tune a rule depending on your needs and enterprise specifics. Writing such rules is not hard and documented in the following tutorial: Custom rules for Java.
There are many different ways how security holes can appear. Logging data to the browser console is only one of them.
And to my knowledge, there is no tool that can detect those security issues automatically. It is the responsibility of the programmer to not expose private user information on a page.
In this case the advice is: Never log passwords (especially unencrypted ones) to the browser console! Instead, encrypt your passwords in the database with an algorithm that can't be decrypted.
Another approach is to create a custom log appender that looks for certain tell-tale patterns (e.g. works like "password" and "passwd") and obliterates the messages, or throws an error.
However, this could be dangerous. If the bad guys knew you were doing this, they might try to exploit it to cover their tracks or even crash your server.
I wouldn't hold my breath for some out-of-the-box solution on this one. Beyond your own logging, you also have to be concerned about the logging done by your dependencies. That said, you have two areas to work on: what goes into the logs and who has access to the logs.
As far as what goes into the logs, your best tools to combat this problem are education and collaboration (including the aforementioned code reviews). Start with writing a list of non-functional requirements for logging that includes security that addresses what to log and how to log (markers, levels, sensitive parameters, etc). I recommend working with colleagues on defining this list so it doesn't become known as "Ravi's logging crusade" instead of "something we really need to do".
Once that list is defined and you get your colleague's and/or management's buy-in, you can write wrappers for logging implementations that support the list of non-functional logging requirements that you assembled. If it is really necessary to log sensitive parameters, provide a way for the parameters to be asymmetrically encrypted for later retrieval by a root account: such as the encryption key stored in a file only accessible by root/container. For management, you might have to spend some time writing up value propositions describing why your initiative is valuable to your company.
Next work with whoever defines your SLDC - make sure the change to your SDLC is outwardly communicated. Have them create a Secure Coding checklist for your company to implement with 1 item on it that says: All logging is implemented using OurCompanySecureLogger. Now you can start working on enforcing the initiative. I recommend writing a check on the build server that looks at dependencies and fails the build if it finds a direct reference to log4j, slf4j, logback, etc.
Regarding the other half of the problem, work with your SysOps team to define rules of Segregation of Duties. That is, software engineers shouldn't have access to the servers where logging is being performed. If you're not staffed well enough at this point to support this notion, you might have to get creative.
May be you should try Contrast tool. Its good one and we are using it since long.
It takes care of all updated owasp top 10 issues.
Quite good for finding security holes in enterprise applications.
Their support is also good.
In my application, I am using Log4j2 in Async mode (Async logger using Disruptor), can some one pls tell me how to handle an exception that happens in the logging process. Logging is very critical for my application, I don't want to miss a single log statement. If something goes wrong with logging, that needs to be alerted immediately.
As per Apache Log4j page, suggests to use ExceptionHandler, but I couldn't find a helping topic on how to use this.
Error handling. If a problem happens during the logging process and an
exception is thrown, it is less easy for an asynchronous logger or
appender to signal this problem to the application. This can partly be
alleviated by configuring an ExceptionHandler, but this may still not
cover all cases. For this reason, if logging is part of your business
logic, for example if you are using Log4j as an audit logging
framework, we would recommend to synchronously log those audit
messages. (Note that you can still combine them and use asynchronous
logging for debug/trace logging in addition to synchronous logging for
the audit trail.)
Waiting for suggestions.
I suggest first trying to implement a simple class that implements the ExceptionHandler interface and confirm that it gets called when a problem occurs. Once this is confirmed you can move on to implementing your custom fallback mechanism.
Secondly, it may be difficult to develop a robust fallback: your fallback may not be able to write to disk if the reason why log4j failed was that the disk is full or damaged. Similarly you may not be able to make a network connection... I suggest that your fallback incorporates multiple options to increase the probability of one of these options succeeding.
Hardware is cheap, so consider adding a separate network card or a separate hard disk for your fallback mechanism. Don't forget to send notifications if the fallback mechanism is used so you can address the original problem.
Depending on how mission-critical this is you may want to investigate vendor products that give high availability - this usually covers more than just logging, not sure what your needs are here.
I am using SLF4J as logging facade and let users decide where and what to log. Now in case of a crash, I want to send a file to the server that contains debugging information--which basically means a log-file. And since we already have all that log-statements scattered in the code, why not use them?
So basically, I want to create a log file programmatically via SLF4J, transparent for the user who still can plug in his own logging backend and configuration.
My first idea was to implement the org.slf4j.impl.StaticLoggerBinder, deliver my own implementation of a logger that does its logging and then delegates to the user-configured logger. However, I see certain issues with this: If the user puts a normal logging backend, then multiple instances of org.slf4j.impl.StaticLoggerBinder are on the classpath. This will issue a warning AND I might not be able to make sure, that my implementation is the one to get called.
Are there better solutions to this? A whole different approach? Is the idea inherently bad? How to accomplish this?
The point of SLF4J is not to let application end-users choose their logging framework (why would they care?) but to let developers include a library without being tied into the library's choice of logging framework.
So if you want to upload debug information from a deployed application, it's fine to fix the logging implementation. The user can still edit the implementation's configuration file, if they want.
Since SLF4J is open-source, you can modify it to use another class than org.slf4j.impl.StaticLoggerBinder. Then your custom StaticLoggerBinder class could load the original, user-provided org.slf4j.impl.StaticLoggerBinder (if it exists).
Another idea is using a custom LoggerFactory (not org.slf4j.LoggerFactory) in your application which returns a Logger delegate. This delegate class delegate logging method calls to the original Logger implementation and also sends the logs to the server if it is necessary.
Anyway, both would look an awkward hack to me, creating two artifacts (one for end-users and another one for developers) smells better.
(Finally, I don't know what kind of library/application it is, but in my working environment it would not be acceptable if a library sends data to a third party server. Are you sure that you really need to to do this?)
i am currently working on a java application for some network monitoring tool. In my code i am supposed to use logging a lot. Since its a network management software, the information in logs is quite useful to the user hence its compulsory to use them. But now I am bit confused with what kind of logger method i should prefer. Right now i am using Logger.lop(...//...) since with its help we are also logging the class name and method so its becoming very easy for me (developers) to debug the code and find the error. But finally I am confused should i deliver it to the end user with the same logging mechanism??? Is it any harm to let your user know what kind of class is executing currently , in which method error has occured. I have seen many times in many product in exception handling stacktrace is used so normally we get class name as well. So is there is no problem to let enduser know what your class name and method is??
Before considering the security implications of it, consider the performance. In most logging systems, getting the actual classname and method name dynamically by the logging facility requires reflection and dramatically slows down the logging - usually a synchronous operation. My guess is that in a network monitoring application, you really don't want that.
If you're hard-coding the method name into the log message (either by making it part of the message or by the category), that's a different story. As a security person, I don't consider it to be that big of a deal - if your code is in Java, it can be reversed anyhow, so your code should operate in such a way that it would be secure even if the code was given away.
All that being said, you could either use a different logging configuration for development and production, or those fine-grained messages could go in debug, trace, etc. If you're using log4j, it's generally advisable to use isDebugEnabled to wrap any logging statements which include anything dynamically-calculated as those get calculated before the logging statement determines whether it's enabled.
log4j/logback/slf4j allow you to have different formats for different appenders. For development you can enable a console appender where you include the class name in the format, while for the end-users you can omit it (for a file appender)
It's worth mentioning that such logging is performance costly in Java, contrary to C++ where it is usually implemented with preprocessor. Fortunately, with log4j/logback you can switch it on and off — follow Bozho's advice.
What's the advantage of log4j over set System.out and System.err to output to a log file?
At a high level, the win from Log4j over manual logging is that you can decouple your logging code from what you actually want to log and where and how you want to log it. Details about logging verbosity/filtering, formatting, log location, and even log type (files, network, etc.) are handled declaratively using configuration and extensibly via custom appenders, rather you having to code that flexibility yourself.
This is critically important because it's often hard for developers to predict how logging needs will change once their software is in production. Operations teams managing that software may need less verbose logs, may need mulitple logs, may need to ship those logs to multiple servers, may need to sometimes get really verbose data for troubleshooting, etc. And it's usually impossible for operations teams, if they need to change how logging works, to convince the developer to make big code changes. This often leads to production downtime, friction between operations and development, and wasted time all around.
From the developer's point of view, Log4j insulates you from having to make code changes to support logging, and insulates you from being pestered by people who want logging changes. It enables people managing your code to scratch their own itch rather than bugging you!
Also, since Log4j is the de-facto standard for Java logging, there are lots of tools available which can do cool things with Log4j-- furthermore preventing you and your operations teams from re-inventing the wheel.
My favorite feature is the ability to easily write appenders send data to non-file sources, like SYSLOG, Splunk, etc. which makes it easy to your app's custom logging into operations management tools your IT department is already using.
Actually, you should look into the slf4j facade these days, as it allows you to use {}-placeholders for the most concise statements. You can then use the appropriate logging framework behind slf4j to handle the actual treatment of your log statements. This could be log4j or the slf4j-simple which just prints out all of INFO, WARN and ERROR, and discards the rest.
The crucial observation you need to make is that the WRITING of log statements is done when the code is written, and the DECISION of what is needed is done when the code is deployed, which may be years after the code was written and tested. System.out.println requires you to physically change your code to get rid of them, which is unacceptable in a rigid write-test-deploy cycle. IF the code changes, it must be retested. With slf4j you just enable those you want to see.
We have full logging in the test phase, and rather verbose logging in the initial period of a production deployment, after which we go down to information only. This gives us full information in a scenario where debugging a case is very rarely possible.
You might find this article I wrote interesting. The target audience is beginning Java programmers, with my intention of giving them good habits from the start. http://runjva.appspot.com/logging101/index.html
my favorites (not all)
Ability to set parameters of logging in config, without recompiling
Ability to set the way log is written (from text file to SMTP sender)
Ability to filter by severity
Levels, formatting, logging to multiple files... A logging framework (even if it's java.util.logging) is really beneficial if there's a chance anything may go wrong while your code is running.
log4j allows you to log to various resources e.g. event log, email, file system etc while allowing your application to remain decoupled from all of these resources. Furthermore, you get to use a common interface to log to all of the various resources without having to learn or integrate thier corresponding APIs.
Log4j offers the ability to rotate your log files based on size and delete them based on quantity (logrotate), so your servers don't fill up their disks. Personally I think that is one of the more valuable features in Log4j.
Also Log4j is popular and understood by many developers. The last three companies I've worked at have all used Log4j in most projects.
Take a look and you will understand the power of log4j :
log4j.properties I used once for a project :
# ALL < DEBUG < INFO < WARN < ERROR < FATAL < OFF
# No appenders for rootLogger
log4j.rootLogger=OFF
folder=..
prefix=
fileExtension=.log
htmlExtension=${fileExtension}.html
datestamp=yyyy-MM-dd/HH:mm:ss.SSS/zzz
layout=%d{${datestamp}} ms=%-4r [%t] %-5p %l %n%m %n%n
# myLogger logger
log4j.logger.myLogger=ALL, stdout, infoFile, infoHtml, errorFile
# stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=${layout}
# infoFile
log4j.appender.infoFile=org.apache.log4j.FileAppender
log4j.appender.infoFile.File=${folder}/${prefix}_info${fileExtension}
log4j.appender.infoFile.layout=org.apache.log4j.PatternLayout
log4j.appender.infoFile.layout.ConversionPattern=${layout}
# infoHtml
log4j.appender.infoHtml=org.apache.log4j.FileAppender
log4j.appender.infoHtml.File=${folder}/${prefix}_info${htmlExtension}
log4j.appender.infoHtml.layout=org.apache.log4j.HTMLLayout
log4j.appender.infoHtml.layout.Title=Logs
log4j.appender.infoHtml.layout.LocationInfo=true
# errorFile
log4j.appender.errorFile=org.apache.log4j.FileAppender
log4j.appender.errorFile.File=${folder}/${prefix}_error${fileExtension}
log4j.appender.errorFile.layout=org.apache.log4j.PatternLayout
log4j.appender.errorFile.layout.ConversionPattern=${layout}
# APPENDERS SETTINGS
log4j.appender.stdout.Threshold = ALL
log4j.appender.infoFile.Threshold = INFO
log4j.appender.infoHtml.Threshold = INFO
log4j.appender.errorFile.Threshold = WARN.
To change the variables in your java code you can do :
Loading Configuration
Log4j will automatically load the configuration if it is stored in a
file called "log4j.properties" and is present on the classpath under
"" (e.g. WEB-INF/classes/log4j.properties).
I don't like that approach and prefer to load the configuration
explicitly by calling:
PropertyConfigurator.configure( Config.ETC + "/log4j.properties" );
This way I can reload the configuration at any time as long as my
application is still running. I like to add a button to an
administrative jsp, "Reload Log4J".
Dynamic Log File Location
Many people complain that Log4j forces you to hard-code the location
where your logs will be kept. Actually, it is possible to dynamically
choose the log-file location, especially if you use the ${log.dir}
property substitution technique above. Here's how:
String dynamicLog = // log directory somehow chosen...
Properties p = new Properties( Config.ETC + "/log4j.properties" );
p.put( "log.dir", dynamicLog ); // overwrite "log.dir"
PropertyConfigurator.configure( p );
logging (Document historical business events that occur, you can check old logs)
track the application (project flow)
debugging the application (Detailed information what occurs in a method at granular level //data, value and all inside methods)
error handling (information about specific error that occur)