We have an application at work that we'd like to monitor for performance. Actually, what we want to monitor is not our app's performance, but things like response time for external web services we invoke.
Years ago, using ATG Dynamo, you could instrument your code with something like...
Performance.monitorStart("my.operation");
try {
// code goes here
}
finally {
Performance.monitorEnd("my.operation");
}
; this generated a nice report of the time spent in diverse operations, in a friendlier way than hprof. Ideally, the time should be persisted (db or otherwise).
I recall seeing somewhere (here? Dzone? TSS?) about a new library that does this, but googling reveals nothing.
Thoughts?
Alex
What you're describing sounds a lot like Perf4J.
Springsource TC Server (which is a Tomcat++) with Insight enabled has been helpful to me
It will time your entire call-stack and give you nice reports. Here's a screencast http://www.youtube.com/watch?v=nBqSh7nVNzc
Since you are already showing an example where code changes are involved, you could simply roll your own using your probably existing logging facility.
Another option would be JMX beans for live statistics - this option is often used together with a 'professional' monitoring facility which aggregates these statistics.
Related
Is there something out there that would allow me to create static object in my class or extend my class and give me functions to start, stop time configure statistics collection with properties file and bunch of other goodies I don't even know about.
I'm working on app that has crazy amount of threads running at any given moment and making sense out of the log files is becoming increasingly difficult. That's why I'm looking for some kind of a solution to help me with that.
Ideally I would like to have bean in my spring application context that would pretty much automate all the tracking of running treads based on annotation which would allow to configure the names of threads and accuracy of the stopwatch. Also ability to hooking it up with database instead of just log file would be great as well.
Maybe you want the apache commons StopWatch class?
Since you are already using Spring, you could use the Spring StopWatch, though it does not meet your additional goals of being able to persist the data. Note that it is not intended for production use, just for additional information during development and testing.
I'm not entirely sure I understand what you are trying to achieve.
Are you looking for keeping statistics on some time-line and exporting those to a a data source using some plugin API?
It might be worth your while to look at the Stats package of Twitter commons, they just open-sourced it not too long ago, so documentation and examples are still scarce, you will need to understand it mostly on your own, but I think this is what you are after.
Source
Javadoc
Is it possible to dump the complete program execution in java? I have to go through a complete process flow for a execution for a specific input values. Using step over, step into is a bit time consuming and I wanted to find out if any java command dumps the execution?
Maybe you want to have a look at the Chronon Time Travel Debugger.
I haven't tried it out yet, after a long beta period it seems to be now officially available and may satisfy your demands. It's a commercial product, but offers a free time trial.
Another alternative may be the use of debugging to a core file using the jsadebugd utility provided with the JDK. (you can't step forwards and backwards, but you can examine the stack/monitors of all threads which might help you already out)
If you only need the method calls, as stated in a comment, maybe a profiler which uses instrumentation like jprofiler or yourkit will also be helpful.
Or you want to have a look at btrace, a dtrace-like tool.
If you're able to modify/build the application, also some sort of a small AOP method interceptor will do the job.
If I understand correctly, you want something like a view of all the method calls that happen when your program processes some set of inputs. You can often get this kind of information out of a profiler, such as JProbe:
http://www.quest.com/jprobe/
You can run the program under JProbe, and then it will present a visual call graph of all of the method calls or a list of all method calls along with their frequency of execution.
Somewhat related are static analysis tools, such as Understand:
http://www.scitools.com/
Static analysis tools tend to focus on figuring out overall code structure rather than what happens with a specific set of inputs though.
Of course, you can always change code, but it's probably too much work to change every method in a large system to print a debugging string. Aspect-oriented programming tends to be a good approach for this kind of problem, because it's a cross-cutting concern across the codebase. There are a few different Java AOP solutions. I've used Spring AOP with dynamic proxies, which isn't enough to cover all method executions, but it is good enough for covering any method execution defined on an interface for a bean managed in a Spring container:
http://static.springsource.org/spring/docs/3.1.0.M1/spring-framework-reference/html/aop.html
For example, I've written a TimingAspect that wraps the execution of a method and logs its execution time after it completes. When I want to use it, I update my Spring applicationContext.xml to specify pointcuts for the methods I want to measure. You could define a similar TracingAspect to print a debugging message at the start of each method execution. Just remember to leave this off for production deployment.
For all of these approaches, measuring every single method call is probably going to cause information overload. You'll probably want to selectively measure just a few important pieces of your own codebase, filtering out core JDK methods and third-party libraries.
I'm going to write my first Java based web app, and I'm sort of lost how to begin.
Firstly, I would like a web app and a desktop app that do pretty much the same thing, without the hackish idea of embedding a web browser into the desktop app because that doesn't allow to easily make changes to the desktop without affecting the web app and vice versa.
Now, here my questions.
Right now, I have a bunch of POJOs and they communicate with a single class that, right now, uses a flat file as a "database", of course, in production, I would use a legitimate database and just change that single class. Is this a good idea? Will I be able to go from POJOs to a web app?
Should I use a framework? I would like to have this app written pretty soon, seeing that all the buisness logic is there, I just need to wrap it so its usable, so, I don't want to spend an extreme amount of time learning, say, Spring (which AFAIK is huge), but, I don't want to keep reinventing the wheel throughout my app either. I can always just use JSP and scriptlets...
If you said yes to the above, what framework(s) do you suggest? Please note that I would like a framework that I can start using in maybe 3-4 weeks of learning.
Will I have to start from scratch with the POJOs that I have written? They're well over 30k LOC, so, if it is like that, I'll be hesitant.
You will need:
a web framework. Since you have Swing background, JSF 2 will be your best bet (everything will be painful, of course, but JSF will get you up and going quickly and will help you avoid the most tragic mistakes). Also, wrapping business pojos into web guis is the main use-case for JSF and it's biggest focus.
a "glue framework". One thing that is much different with web applications as opposed to desktop ones is that you cannot create view components by yourself - they must be created when browser requests a page. So you have to find a way to create the view objects and deliver all the references to the pojos that represent logic, some of which may have very different lifecycles (this is not a problem on desktop, but on web you have to distinguish between pojos that live along with the whole application, along with a single user session, along with a single request, and so on).
The "glue framework" could also provide the additional benefit of managing transactions. You have three choices:
Spring. It's not half as complex as you thing; you only need to learn some basic stuff.
EJB. You would need a real application server, like Glassfish or JBoss
bare JSF has good support for dependency injection, the only drawback is the lack of automatic transaction management.
If I were in your position, I would go with bare JSF 2.0 - this way you only need to learn one new technology. At first, try to avoid libraries like PrimeFaces - they usually work worse than advertised.
edit - and addendum
or - what is "dependency injection"(abridged and simplified)
When request comes to a web application, a new task starts in a new thread (well, the thread is probably recycled, but that's not important).
The application has already been running for some time and most of the objects you are going to need are already built and should not get created again: you have your database connection pool, maybe some parts of business layer; it is also possible that the request is just one of many request made during one session, and you already have a bunch of POJOs that the user is working on. The question is - how to get references to those objects?
You could arrange your application so that resources are available through some static fields. They may be singletons themselves, or they could be acquired through a singleton locator. This tends to work, but is out of fashion (hard to test, hard to refactor, hard to reuse, lifecycles are hard coded in application). The real code could look like this:
public void doSomething() {
Customer Service cs = AppManager.getInstance().getCustomerService();
System.out.println(cs.getVersion());
}
if you need clustering and session management, you could build a special kind of broker that would know and provide to anyone all kinds of needed objects. Each type of object would be registered as a factory under a different name. This also works and is implemented in Java as JNDI. The actual client code would look like this:
public void doSomething() throws Exception {
CustomerService cs = (CustomerService)new InitialContext().lookup("some_fancy_looking_name_in_reality_just_string");
System.out.println(cs.getVersion());
}
The last way is the nicest. Since your initial object is not created by you but by the server just after http request arrives (details depend on the technology you choose, but your entry point might be a JSF managed bean or some kind of action controller), you can just advertise which references you need and let the server take care of finding them for you. This is called "Dependency Injection". Your acts as if everything is taken care of before your code is ever launched. Spring or EJB container, or CDI, or JSF take care of the rest. The code would look like this (just an example):
#EJB
CustomerService cs;
public void doSomething() {
System.out.println(cs.getVersion());
}
Note:
when you use DI, it really uses one of the two former methods under the hood. The good thing is: you do not have to know which one and in some cases you can even switch them without altering your code;
the exact means of registering components for injection differs from framework to framework. It might be a piece of Java code (like in Guice), an XML file (classic Spring) or an annotation (classic EJB 3). Most of the mentioned technologies support different kinds of configuration.
You should definitely use a framework as otherwise sooner or later you'll end up writing your own.
If you use maven then simply typing mvn archetype:generate will give you a huge list of frameworks to choose from and it'll set up all of the scaffolding for you so you can just play with a few frameworks until you find the one that works for you.
Spring has good documentation and is surprisingly easy to get started with. Don't be put off by the pages of documentation! You could use JPA to store stuff in the database. You should (in theory) just be able to annotate your existing POJO's to denote primary keys and so on and it should just work. You can also use JSP's within Spring if that makes life easier.
... I a bunch of POJOs and they communicate with a single class that, right now, uses a flat file as a "database", of course, in production, I would use a legitimate database and just change that single class. Is this a good idea? Will I be able to go from POJOs to a web app?
qualified yes. if the pojo's are sane you should not have many problems. many people use hiberbate.
Should I use a framework? I would like to have this app written pretty soon, seeing that all the buisness logic is there, I just need to wrap it so its usable, so, I don't want to spend an extreme amount of time learning, say, Spring (which AFAIK is huge), but, I don't want to keep reinventing the wheel throughout my app either. I can always just use JSP and scriptlets...
probably. spring is huge, but things like grails or roo can help.
if you want to have a responsive web app, you will need to do some kind of rich client (AJAX). this may require a lot of your code to run on the client. this means writing a lot of javascript or using gwt. this will be a pain. it probably will not be so easy to just "wrap it". if you have written a swing app, then basically that code will need to run on the client.
If you said yes to the above, what framework(s) do you suggest? Please note that I would like a framework that I can start using in maybe 3-4 weeks of learning.
i like groovy and grails - grails uses spring-mvc, spring, hibernate. but there is roo, play and others.
Will I have to start from scratch with the POJOs that I have written? They're well over 30k LOC, so, if it is like that, I'll be hesitant.
the code that will run on the server can probably be mostly left alone. the code that has to run on the client needs to be rewritten in javascript or maybe you can get some reuse out of that code by using gwt,
The Play Framework is doing great things. I would recommend it highly. Having worked with EJB apps and Tomcat/Servlet/Spring apps it's a breath of fresh air. After framework installation you get a working app in a few seconds. Reminds me of Ruby on Rails or Node.js with the type-safety of Java.
Much quicker turnaround on getting started, faster development cycles, and a clearer configuration model than previous Java web app frameworks.
http://www.playframework.com/
Aside from adding a custom timer to measure the beginning and end of a controller's action, is there an easier or more helpful way to show how long a page is really loaded (i.e. show at the bottom of the page: this page is generated in 30.5 seconds)? Note that in Grails, there's the concept of taglibs wherein you can add additional logic after all the processing done in the controller.
I'm actually not yet sure on how controller and taglib works or how the whole page is rendered in Grails, perhaps they are processed in parallel? Feel free to enlighten me on this too.
Thanks!
If you just want the times spent on the action and to render your gsp (with all its tags), you can use a simple filter to measure that. Take a look at this blogpost: Profiling web requests in your Grails application (disclaimer: I'm the author)
Regards,
Deluan
there are several ways to get the timing.
Easiest way is to configure your server to write logs with page generation time
you could inject your timing into for instance a security filter and the end of your page - but as you already mentioned, even this would mean to reinvent the wheel
have you checked the plugins?
debug plugin gives you timing info on the console http://www.grails.org/plugin/debug
perf4j also helps to profile your pages http://www.grails.org/plugin/perf4j
but if you want to present the timing to your users, I would suggest to download the debug plugin, unzip it and check where the timing is measured. You can easily copy this code and use it to output the timing on the page.
I used Spring Insight with STS. That's absolutely awesome for Grails Application in developpement. Modifying a tomcat for use in poroduction make it a bit tricky though
But you can go down to the duration of each select from hibernate and you have timing metric in real time through the all stack of application
Not really what you asked for (sorry), but maybe of interest is the JavaMelody plugin for Grails:
The goal of JavaMelody is to monitor
Java or Java EE application servers in
QA and production environments. It is
not a tool to simulate requests from
users, it is a tool to measure and
calculate statistics on real operation
of an application depending on the
usage of the application by users.
Not tried it myself, but it looks useful
I have a project right now that straddles the line on framework and pluggable program, and am worried about the sheer number of dependancies that this program rely's on.
Currently I have this:
Commons lang - Mainly for string utils and array utils
slf4j - Logging facade
slf4j-log4j - Redirects logging to log4j for GUI (note that the GUI is a module)
log4j - Log4j itself for the above reason
jpersist/EJP - Database abstraction layer
PircBot - IRC layer
A JDBC driver
Mozilla Rhino - For Javascript plugins
In all that totals 7, even without the GUI unless you don't want any logging. For me who's trying to pass this off as "lightweight", this seems like way too much.
So my questions:
Should I limit the amount of frameworks that I am using?
How should I distribute it? Would an independent jar for being used in other programs and a big combined jar for a single program be okay?
Is this many dependencies normal?
It does seem like quite a lot. Regardless of the issue of specifying numerous libraries, you're restricting your users wrt. the third-party libraries they can use in their project to the ones you specify.
Can you specify implementation-agnostic libraries ? e.g commons-logging, which will delegate to existing logging frameworks under the covers. If your users are already using something other than log4j, then this will permit them to carry on without having to switch.
Secondly, is your framework doing too much ? Instead of providing a chat implementation, why not provide a suitable API such that clients can plug in their own chat/notification mechanism. That way your framework becomes more generic and (again) your clients can choose what/how to implement features. A rich client API will give your users many more options and extend the usefulness of your framework.
Should I limit the amount of frameworks that I am using?
If you are really using/needing them, not really. I would just try to avoid overlapping libraries and adding a library if you're only using 1% of it.
How should I distribute it? Would an independent jar for being used in other programs and a big combined jar for a single program be okay?
Many projects are distributed as a zip/tar.gz distro. But for a framework, making it available as a Maven artifact would be a great plus (in which case, make log4j and the log4j binding optional).
Is this many dependencies normal?
Firstly, you don't have that many dependencies. Secondly, there is IMO nothing wrong with reusing a logging facade, a persistence library, utility classes, etc (not using such libraries and writing your own code to replace them would be stupid). Thirdly, most users don't care, especially if you are delivering nice features (instead of spending time reinventing the wheel and, ultimately, creating bugs).
the scale of your project ie what it accomplishes and in what environment it will be used will balance out against how many dependencies and ease of configuration in general when people assess the suitability of your project.
you haven't really hinted at what your project attempts to achieve so it's difficult to say whether you have a bloated stack. however, for something reasonably useful I personally wouldn't have a problem with most of those jars.
the only thing that rings alarm bells is the database layer and the jdbc driver. if your project is a 'framework' i fail to see how a particular jdbc driver fits the model, and persistence in general does not quite fit the model of a framework.
That might seem like a lot of dependencies, but I don't think it is in reality. Certainly, there doesn't seem to be much gratuitous duplication of functionality. Most of the dependencies are doing things that you'd otherwise need to implement yourself.
For me who's trying to pass this off as "lightweight", this seems like way too much.
Maybe you need to adjust your rhetoric. :-)
Seriously, if those dependencies are necessary, the only way you will be able to get rid of them is to either code equivalent functionality yourself (bad idea) or drop the corresponding functionality (maybe a bad idea). Which is more important to you; being lightweight or being functional?
EDIT
Functional is key in the end. I can have my own custom implementation of everything but it would be full of bugs I guess. However I would like to keep it small as small and easy do attract people.
Well you clearly understand the issues. The decision is yours to make. (But don't forget that while some people are put off by "bloat", others are attracted by lots of functionality.)
I suppose that there is a half-way solution. Keep the functionality, but make it optional and provide some way that people can configure it in / out. Of course, the downside is that this means that you have to test multiple permutations of configuration options, and it makes installation / configuration more complicated for your users.
I think you worry to much :) The number of dependencies is not relevant, the maturity of them it is. If you will drop functionality/usability/flexibility/etc just because you want to keep the number of dependencies "low" it would be a loss for you (and your clients).