I am having a fairly large legacy (swing) application and currently brainstorming and evaluating the strategies to add logging in it.
Key point is that it's fairly large (like I said above) and thinking what could be the strategy to do it quickly and completely (100% coverage for argument sake) (let's say logging is no customer facing core business functionality, so let's say not getting paid for it)
So the two strategies as such are -
Do use log4j2 add the debug statements, add appenders, file rolling rules, add dynamic on/off and change of log levels etc and thus do it the way it ideally should be done.
Use AOP to define endpoints and advices declaratively/programmaticly. Device such elaborate rules, add in the rest of the part (appenders, rolling rules etc.) and thus wont be touching each and every file and adding a revision just for that.
The second strategy is what I want to explore if someone has experience with, if somebody has gone that route.
The cons I can think of -
No standard / convention in apps, would be 99% impossible to define endpoints completely
Sphagetti and unmaintainable code in AOP module above, largely a result of the above problem
Adding same and matching set to logging to future classes.
Can you guys suggest me wither ways
While I am a big proponent of refactoring and clean code, I suggest you use AspectJ to add logging and see how far you get. Probably you will be surprised by how much logging you get with a little aspect.
IMHO logging is a cross-cutting concern and not core functionality. I would try to avoid logging in my core code. My expecience with AspectJ is only positive, I warmly recommend it. You should take some time to get acquainted with its power, though.
If you need something like a call trace (maybe neatly indented) or just plain method call logging with parameters and maybe return values, you will get quick results with AspectJ without cluttering your application code. The aspect can always be improved and maintained in a single spot, which is very clean. No need to be afraid of spaghetti aspect code. It will not happen if you don't totally abuse AspectJ.
You will only run into problems if your core code is already spaghetti code and you have very long methods which need log statements inside those methods and not around them. Then you might need to refactor and break down monster methods into smaller ones. But that is a problem no matter which logging approach you choose. My own experience with ugly legacy code and AspectJ-based logging tells me that it helps a lot because even spaghetti code calls other methods and that can be nicely traced in AspectJ.
Related
I have been working on a project that logs using what are essentially just println statements with a prefixed string tag. I have been looking into implementing support for an actual logging library such as Logback the past few days and had some questions relating to best-practices about logging in general. I know a lot of what I'm doing is probably stupid, but I want to change :)
When I'm extending the code and adding new features, such as testing a new codec, I have been using liberal logging to ensure the code behaves as expected (instead of actual unit tests), and then using constant booleans at the top to disable that logging when the codec is finished (in case it's needed again or a bug is found, I can flip the boolean while testing). I don't know if the granularity that debug level provides would be enough and would prefer some way to define levels differently for different features. Leaving these enabled by default would really bloat the console and probably effect performance -- is this what filters are usually used for?
I've also found myself in more than one case prepending spaces to my messages so that I can better follow the flow of the code. I've found this to be really helpful. In a way, the tabbed messages are like a debug-debug level.
Doing something
Reading a file
header of file: ...
body of file: ...
Back at main
What are good practices for logging? Can someone refer me a good resource that I can dig into or explain if what I'm doing is stupid and why it's stupid? What are some alternatives? An open source project as an example would be extremely helpful. Thanks, I appreciate any guidance.
Some advice:
Never substitute unit tests for logging only.
Regarding logging you should log whatever makes you find bugs quicker.
Log libraries support async logging which will not affect the performance of your application (log4j2 async logging). Logback supports async too.
Do not use booleans inside your code to decide to log or not to log. Use the logging levels (TRACE, DEBUG, INFO, WARN, ERROR) and set them accordingly. Usually in PROD environment you will use a WARN level and in DEV you can set it on debug.
Logging on different levels depending on the package is quite useful (you can use appenders to customize this and other stuff).
In resume, the important is to read the documentation of whatever library you are using
how do you go about structuring logging in your java projects?
I have used log4j2, logback etc, however this is not a question about libraries but about setup and best practices.
I have tried:
in filter
but it does not have enough detail. For example i cannot log method calls, input/output values in methods etc. Also it does not feel good when i need detailed logs of user activities.
in each class
In each class, a static field for the logger, then in methods where needed i log what i need to. This way i have all the detail i want, but it's a lot pf work and it feels like the code is littered with log stuff.
What do you use in your projects?
Thanks
I generally go with a similar approach to that of using static loggers in Class, but with a twist. Generally, I use only one static logger and a collection of Aspects. When a method requires to be logged, and usually it's input/output that needs to be logged, I simply use an #Around aspect. This keeps your code clean and separates the concerns of business logic with the concerns of the programmer (logging).
Here's a great place to start: http://docs.spring.io/spring/docs/current/spring-framework-reference/html/aop.html
i am currently working on a java application for some network monitoring tool. In my code i am supposed to use logging a lot. Since its a network management software, the information in logs is quite useful to the user hence its compulsory to use them. But now I am bit confused with what kind of logger method i should prefer. Right now i am using Logger.lop(...//...) since with its help we are also logging the class name and method so its becoming very easy for me (developers) to debug the code and find the error. But finally I am confused should i deliver it to the end user with the same logging mechanism??? Is it any harm to let your user know what kind of class is executing currently , in which method error has occured. I have seen many times in many product in exception handling stacktrace is used so normally we get class name as well. So is there is no problem to let enduser know what your class name and method is??
Before considering the security implications of it, consider the performance. In most logging systems, getting the actual classname and method name dynamically by the logging facility requires reflection and dramatically slows down the logging - usually a synchronous operation. My guess is that in a network monitoring application, you really don't want that.
If you're hard-coding the method name into the log message (either by making it part of the message or by the category), that's a different story. As a security person, I don't consider it to be that big of a deal - if your code is in Java, it can be reversed anyhow, so your code should operate in such a way that it would be secure even if the code was given away.
All that being said, you could either use a different logging configuration for development and production, or those fine-grained messages could go in debug, trace, etc. If you're using log4j, it's generally advisable to use isDebugEnabled to wrap any logging statements which include anything dynamically-calculated as those get calculated before the logging statement determines whether it's enabled.
log4j/logback/slf4j allow you to have different formats for different appenders. For development you can enable a console appender where you include the class name in the format, while for the end-users you can omit it (for a file appender)
It's worth mentioning that such logging is performance costly in Java, contrary to C++ where it is usually implemented with preprocessor. Fortunately, with log4j/logback you can switch it on and off — follow Bozho's advice.
Is it possible to dump the complete program execution in java? I have to go through a complete process flow for a execution for a specific input values. Using step over, step into is a bit time consuming and I wanted to find out if any java command dumps the execution?
Maybe you want to have a look at the Chronon Time Travel Debugger.
I haven't tried it out yet, after a long beta period it seems to be now officially available and may satisfy your demands. It's a commercial product, but offers a free time trial.
Another alternative may be the use of debugging to a core file using the jsadebugd utility provided with the JDK. (you can't step forwards and backwards, but you can examine the stack/monitors of all threads which might help you already out)
If you only need the method calls, as stated in a comment, maybe a profiler which uses instrumentation like jprofiler or yourkit will also be helpful.
Or you want to have a look at btrace, a dtrace-like tool.
If you're able to modify/build the application, also some sort of a small AOP method interceptor will do the job.
If I understand correctly, you want something like a view of all the method calls that happen when your program processes some set of inputs. You can often get this kind of information out of a profiler, such as JProbe:
http://www.quest.com/jprobe/
You can run the program under JProbe, and then it will present a visual call graph of all of the method calls or a list of all method calls along with their frequency of execution.
Somewhat related are static analysis tools, such as Understand:
http://www.scitools.com/
Static analysis tools tend to focus on figuring out overall code structure rather than what happens with a specific set of inputs though.
Of course, you can always change code, but it's probably too much work to change every method in a large system to print a debugging string. Aspect-oriented programming tends to be a good approach for this kind of problem, because it's a cross-cutting concern across the codebase. There are a few different Java AOP solutions. I've used Spring AOP with dynamic proxies, which isn't enough to cover all method executions, but it is good enough for covering any method execution defined on an interface for a bean managed in a Spring container:
http://static.springsource.org/spring/docs/3.1.0.M1/spring-framework-reference/html/aop.html
For example, I've written a TimingAspect that wraps the execution of a method and logs its execution time after it completes. When I want to use it, I update my Spring applicationContext.xml to specify pointcuts for the methods I want to measure. You could define a similar TracingAspect to print a debugging message at the start of each method execution. Just remember to leave this off for production deployment.
For all of these approaches, measuring every single method call is probably going to cause information overload. You'll probably want to selectively measure just a few important pieces of your own codebase, filtering out core JDK methods and third-party libraries.
I have a project right now that straddles the line on framework and pluggable program, and am worried about the sheer number of dependancies that this program rely's on.
Currently I have this:
Commons lang - Mainly for string utils and array utils
slf4j - Logging facade
slf4j-log4j - Redirects logging to log4j for GUI (note that the GUI is a module)
log4j - Log4j itself for the above reason
jpersist/EJP - Database abstraction layer
PircBot - IRC layer
A JDBC driver
Mozilla Rhino - For Javascript plugins
In all that totals 7, even without the GUI unless you don't want any logging. For me who's trying to pass this off as "lightweight", this seems like way too much.
So my questions:
Should I limit the amount of frameworks that I am using?
How should I distribute it? Would an independent jar for being used in other programs and a big combined jar for a single program be okay?
Is this many dependencies normal?
It does seem like quite a lot. Regardless of the issue of specifying numerous libraries, you're restricting your users wrt. the third-party libraries they can use in their project to the ones you specify.
Can you specify implementation-agnostic libraries ? e.g commons-logging, which will delegate to existing logging frameworks under the covers. If your users are already using something other than log4j, then this will permit them to carry on without having to switch.
Secondly, is your framework doing too much ? Instead of providing a chat implementation, why not provide a suitable API such that clients can plug in their own chat/notification mechanism. That way your framework becomes more generic and (again) your clients can choose what/how to implement features. A rich client API will give your users many more options and extend the usefulness of your framework.
Should I limit the amount of frameworks that I am using?
If you are really using/needing them, not really. I would just try to avoid overlapping libraries and adding a library if you're only using 1% of it.
How should I distribute it? Would an independent jar for being used in other programs and a big combined jar for a single program be okay?
Many projects are distributed as a zip/tar.gz distro. But for a framework, making it available as a Maven artifact would be a great plus (in which case, make log4j and the log4j binding optional).
Is this many dependencies normal?
Firstly, you don't have that many dependencies. Secondly, there is IMO nothing wrong with reusing a logging facade, a persistence library, utility classes, etc (not using such libraries and writing your own code to replace them would be stupid). Thirdly, most users don't care, especially if you are delivering nice features (instead of spending time reinventing the wheel and, ultimately, creating bugs).
the scale of your project ie what it accomplishes and in what environment it will be used will balance out against how many dependencies and ease of configuration in general when people assess the suitability of your project.
you haven't really hinted at what your project attempts to achieve so it's difficult to say whether you have a bloated stack. however, for something reasonably useful I personally wouldn't have a problem with most of those jars.
the only thing that rings alarm bells is the database layer and the jdbc driver. if your project is a 'framework' i fail to see how a particular jdbc driver fits the model, and persistence in general does not quite fit the model of a framework.
That might seem like a lot of dependencies, but I don't think it is in reality. Certainly, there doesn't seem to be much gratuitous duplication of functionality. Most of the dependencies are doing things that you'd otherwise need to implement yourself.
For me who's trying to pass this off as "lightweight", this seems like way too much.
Maybe you need to adjust your rhetoric. :-)
Seriously, if those dependencies are necessary, the only way you will be able to get rid of them is to either code equivalent functionality yourself (bad idea) or drop the corresponding functionality (maybe a bad idea). Which is more important to you; being lightweight or being functional?
EDIT
Functional is key in the end. I can have my own custom implementation of everything but it would be full of bugs I guess. However I would like to keep it small as small and easy do attract people.
Well you clearly understand the issues. The decision is yours to make. (But don't forget that while some people are put off by "bloat", others are attracted by lots of functionality.)
I suppose that there is a half-way solution. Keep the functionality, but make it optional and provide some way that people can configure it in / out. Of course, the downside is that this means that you have to test multiple permutations of configuration options, and it makes installation / configuration more complicated for your users.
I think you worry to much :) The number of dependencies is not relevant, the maturity of them it is. If you will drop functionality/usability/flexibility/etc just because you want to keep the number of dependencies "low" it would be a loss for you (and your clients).