I am wondering if it is possible to store the results shown by visualVM (memory usage, GC activity, Heap...) in a database so that we could consult them later?
If anyone has an idea, or a better alternative to visualVm I would be thankful.
Many thanx.
All these metrics can be obtained using JMX. You can write a monitoring Java program that connects to external (target) JVMs' MBeanServer and dumps selected JMX metrics into the database.
You can also use fabulous Jolokia library to fetch these metrics using virtually any language that support HTTP. Also Jolokia itself can store historical values of selected metrics.
That being said, can't you simply dump interesting values to application logs (or maybe to some selected file) and process them offline? Log files are both easier to maintain (logging framework can delete old ones for you) and to access (storing in database vs. logging to file).
I am not sure what do you mean by 'consult them later', but if it means that you want to do the off-line analysis of collected data by VisualVM, you can create 'Application snapshot' from monitored application. This application snapshot encapsulates data and graphs as well as thread-dumps, heap-dumps and CPU/memory snapshots obtained during application monitoring.
Related
I am dealing with one of the critical software service in our java web application.
We wanted to monitor the java code level statistics(eg. Mean rate, avg rate and number of order received in system) for our web application so that we can understand that what is going on inside the java code and we can take steps early to rectify them.
There are plenty of open source application monitoring API's are available on internet for this purpose. We have looked into Yammer metrics and Jamon API(Source forge).
But there are some common concerns regarding these all open source projects :
If we incorporate these API's in our main application code base, then it will generate statistics data which can acquire memory on the system. This can lead to potential issues like system breakdown or performance degradation.
In our system we need to capture whole statistics data which was generated since the starting. So somehow we need to continuously stream the generated data to some other server so that it is not saved on main system memory.
We always wanted to implement this performance monitoring system which is loosely coupled from our main application code. In short we cannot claim any memory inside the main application server, because that can shut down the whole business.
I searched internet a lot but cannot find an efficient solution.
Using Service Management REST APIs (http://msdn.microsoft.com/en-us/library/windowsazure/ee460799.aspx) we can get list of VM under our subscription.
Now i want to get the details of the underlying OS as well.. like which version of windows is installed, RAM, Processor, free HDD space etc.
Is there any API provided by Microsoft for this ? I am looking for a solution compatible with Java.
Using service management API you can get the size of the role instance (Small, Medium etc). Each size has predefined number of cores, RAM, Hard Disk size etc. so you can find this information that way. Service management API won't give you this information directly.
Am getting to do a project that consists of a cross-platform(windows, linux, mac) native desktop app that measures a user's data usage (in KB, MB, GB) and the link connection speed(in kbps or mbps) for either wi-fi, ethernet or 3G modem connectivity options.
Am confortable with java, python, C++ in that order. Done some googling, but i wasn't satified. So am looking for any recommendations on libraries, frameworks in any of these languages that will enable me accomplish the above task. Thanks alot.
The task can be split into two parts:
Gathering the data and storing them somewhere.
Presenting them to the user.
Often RRDtool (RRD = round-robin database) is used for both parts. It depends on your requirements, you could for example use RRDtool to gather the data you need and them use some Python or Java RRD library for displaying the results.
See also:
Java RRD library
rrd4j
PyRRD
I want to currently profile my application which uses JCS for caching.I have used the default profiler which comes with netbeans IDE and it proves satisfactory for memory.But I want to profile my application for disk usage,because my app caches items in disk I want to verify its performance.Is there a way to do this using the profiler that comes with netbeans or using any commercial profiler such as JProfiler.
Thanks in advance
JProfiler has a file probe that will show you:
a time line of files and the I/O activities on them
a list of all files with their I/O statistics
"hot spots", i.e. files on which most read/write operations are performed together with cumulated back traces to your code
telemetries for total read and write throughput
an "events" view showing single I/O operations
Disclaimer: My company develops JProfiler
You can do this using one of the built-in probes in JProfiler.
I have a Tomcat instance which is exhibiting the following behaviour:
Accept a single http incoming request.
Issue one request to a backend server and get back about 400kb of XML.
Pass through this XML and transform it into about 400kb of JSON.
Return the JSON response.
The problem is that in the course of handling the 400k request my webapp generates about 100mb of garbage which fills up the Eden space and triggers a young generation collection.
I have tried to use the built in java hprof functionality to do allocation sites profiling but Tomcat didn't seem to start up properly with that in place. It is possible that I was just a bit impatient as I imagine memory allocation profiling has a high overhead and therefore tomcat startup might take a long time
What are the best tools to use to do java memory profiling of very young objects/garbage? I can't use heap dumps because the objects I'm interested in are garbage.
As to the actual problem: XML parsing can be very memory hogging when using a DOM based parser. Consider using a SAX or binary XML based parser (VTD-XML is a Java API based on that).
Actually, if the XML->JSON mapping is pure 1:1, then you can also consider to just read the XML and write the JSON realtime line by line using a little stack.
Back to the question: I suggest to use VisualVM for this. You can find here a blog article how to get it to work with Tomcat.
You can use the profiler in jvisualvm in the JDK to do memory profiling.
Also have a look at Templates to cache the XSLT transformer.
http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/transform/Templates.html
You should be able to get heap dumps to work anyway by debugging the app, placing breakpoints at key points of the code and creating a heap dump while the app is paused at each breakpoint.
You might want to try LambdaProbe, which is a profiler for Tomcat.
It supports the following:
Overview
Lambda Probe (formerly Tomcat Probe) is a self sufficient web application, which helps to visualize various parameters of Apache Tomcat instance in real time. Lambda Probe is designed to work specifically with Tomcat so it is able to access far more information that is normally available to JMX agents. Here is a list of features available through Lambda Probe:
New! Comprehensive JVM memory usage
monitor.
JBoss compatibility
Display of deployed applications,
their status, session count, session
object count, context object count,
datasource usage etc.
Start, stop, restart, deploy and
updeploy of applications
Ability to view deployed JSP files
Ability to compile all or selected
JSP files at any time.
Ability to pre-compile JSP files on
application deployment.
New! Ability to view auto-generated
JSP servlets
Display of list of sessions for a
particular application
Display of session attributes and
their values for a particular
application. Ability to remove
session attributes.
Ability to view application context
attributes and their values.
Ability to expire selected sessions
Graphical display of datasource
details including maximum number of
connections, number of busy
connections and configuration details
New! Ability to group datasource
properties by URL to help visualizing
impact on the databases
Ability to reset data sources in case
of applications leaking connection
Display of system information
including System.properties, memory
usage bar and OS details
Display of JK connector status
including the list of requests
pending execution
Real-time connector usage charts and
statistics.
Real-time cluster monitoring and
clulster traffic charts
New! Real time OS memory usage, swap
usage and CPU utilisation monitoring
Ability to show information about log
files and download selected files
Ability to tail log files in real
time from a browser.
Ability to interrupt execution of
"hang" requests without server
restart
New! Ability to restart Tomcat/JVM
via Java Serview Wrapper.
Availability "Quick check"
Support for DBCP, C3P0 and Oracle
datasources
Support for Tomcat 5.0.x and 5.5.x
Support for Java 1.4 and Java 1.5
https://github.com/mchr3k/org.inmemprofiler/wiki (http://mchr3k.github.io/org.inmemprofiler/)
InMemProfiler can be used to identify which objects are collected after a very short time.