I have a java project written in eclipse (RAD, actually); it uses a significant amount of memory by virtue of using iText. I am looking at using a different way of generating my iText document that is supposed to use less memory. I want to know how much less memory it uses.
I know what object will be the root for the largest portion of the memory; it would be fine for my purposes if I could set a breakpoint and then do something that would tell me the deep-copy memory used starting with that object (i.e., the memory used by it and all of its direct and indirect references).
I've been looking at memory monitors and heap dump analyzers and so forth for an hour now, and am pretty confused. All of them appear to be pointed at answering a different problem, or at least pointed to such a general class of problems that, even if I could get them installed and working, it is not clear whether they would answer MY question.
Can someone point me to a reasonably simple way to answer this limited question? I figure if I run the code once the current way, find out the memory used by this object and maybe one or two others, then run it again and look at the same values, I'll know how much good the new technique is doing.
rc
http://www.eclipse.org/mat/
works great for me... tutorials are included
You can fire up JVisualVM, shipped with all Oracle JDK and available independently.
Monitor your process, and at some point, you can do a Heap Dump.
In the heapdump tab, you can go to the OQL console and select your object[s].
When viewing the instance of an object, you can request to compute the retained size. That will give you the total size of your object.
Related
I have a memory leak problem in my GWT application and I'm trying to profile it using JProfiler.
I can't manage to get pertinent results as I don't see my java classes on the profile memory view, i just see the GWT lib classes.
I've added the parameter to profile a remote application using JProfiler (-agentpath:C:\PROGRA~1\JPROFI~1\bin\WINDOW~1\jprofilerti.dll=port=8849). I launch the project in superDevMode through the Eclipse IDE. JProfiler shows me the GWT classes in memory but it doesn't show my own java classes.
In this video youtube.com/watch?v=zUJUSxXOOa4 we can see that JProfiler can show the java classes directly, that's what i search to do
Is there any option to activate in JProfiler for that ? Any help on that matter would be welcome. Thank you
Super Dev mode does not work with a Java profiler. The old Dev mode executed the client-side code in the JVM via a special plugin. These days, the Dev mode browser plugin does not work with modern browsers. The last browsers that supported the plugins were Chrome 21.0.1180.89 and Firefox 26.
As of now, Firefox 24 ESR is still supported:
https://www.mozilla.org/en-US/firefox/organizations/all/
and the Dev mode plugin works in that version. For more information on dev mode see
http://www.gwtproject.org/doc/latest/DevGuideCompilingAndDebugging.html
I don't think so. GWT compile your classes to JS, due to this JProfiler won't work (I think, but maybe I'm wrong). Maybe you can give a try to MemoryAnalyzer with a Heap dump.
Chrome comes with some awesome built-in CPU and heap profiling tools. Firefox now has its own built-in CPU profiler, and Firebug has a different one. IE (at least 10, and I think 9) has a built in CPU profiler, though it has been a long time since I dug too far into there.
Memory is historically a difficult thing to track in browsers, not least because old IE versions just won't die, and leak just from looking at them funny. If you are facing one of those memory leaks, a different plan of attack is required.
But if you suspect you are dealing with a leak in your own application code, Chrome's dev tools can help! Compile in PRETTY (or DETAILED if you have an extremely wide screen), and bring up your app in Chrome with the developer tools open.
In the Profiles tab, there are three kinds of profile to capture, two about memory. I typically prefer the Take Heap Snapshot, and take a 'before' and 'after' look at whatever action I believe to be leaking memory, but the Record Heap Allocations view will give you another way to consider the memory usage of your application.
Start by picking a supposedly 'stable' state of your memory usage - turn the app on, use it for a bit, make sure all the various singletons etc are instantiated, and probably do whatever action you suspect as causing a problem, once. Once you are at a point which you can back to (memory-wise, at least if the leak was fixed), take a snapshot, do the behavior that leaks, return to the 'stable' state, and take another snapshot. Only take one step when checking for leaks, more on this in a bit.
With the two snapshots, you can compare objects allocated and freed - we're mostly interested in cases where more objects were created than deleted, ideally where zero were deleted. If you find N objects are deleted but N+1 are created, then make sure N is veeeery small before digging in - it is often possible to fix a leak only by going after individual objects, tracing them back to their actual leaked source, fixing it, and measuring again.
Once you have an object that was created in one step, but not deleted at the end of that step (but it should have been) use the 'Retainers' view to see why they are still kept. This will more or less show you the field in the object that holds them and the type of that holding object, all the way up to window or some other global object.
Ignore anything in ()s like (compiled code), (array), (system), (string), etc. I'd generally ignore dom element allocation (assuming you suspect you have a leak in app code, not JSNI). Look for few, high level objects leaked, rather than many, low level, it will make it more likely that you are closer to the source of the leak.
The names of compiled constructors and fields in PRETTY generally map very closely to the original Java source. All constructors get _X appended to them, where X is 0, 1, etc - this is to distinguish from the type itself. This makes for an easy way to recognize Java types in the Constructor column, as they all have _s near the end of their name.
My application currently consumes quite a lot of memory because it is running physics simulations. The issue is that consistently, at the 51st simulation, Java will throw an error usually because of a heap space out of memory (my program eventually runs thousands of simulations).
Is there anyway I can not just increase the heap space but modify my program so that the heap space is cleared after every run so that I can run an arbitrary number of simulations?
Edit: Thanks guys. Turns out the simulator software wasn't clearing the information after every run and I had those runs all stored in an ArrayList.
There is no way to dynamically increase the heap programatically since the heap is allocated when the Java Virtual Machine is started.
However, you can use this command
java -Xmx1024M YourClass
to set the memory to 1024
or, you can set a min max
java -Xms256m -Xmx1024m YourClassNameHere
If you are using a lot of memory and facing memory leaks, then you might want to check if you are using a large number of ArrayLists or HashMaps with many elements each.
An ArrayList is implemented as a dynamic array. The source code from Sun/Oracle shows that when a new element is inserted into a full ArrayList, a new array of 1.5 times the size of the original array is created, and the elements copied over. What this means is that you could be wasting up to 50% of the space in each ArrayList you use, unless you call its trimToSize method. Or better still, if you know the number of elements you are going to insert before hand, then call the constructor with the initial capacity as its argument.
I did not examine the source code for HashMap very carefully, but at a first glance it appears that the array length in each HashMap must be a power of two, making it another implementation of a dynamic array. Note that HashSet is essentially a wrapper around HashMap.
There are a variety of tools that you can use to help diagnose this problem. The JDK includes JVisualVM that will allow you to attach to your running process and show what objects might be growing out of control. Netbeans has a wrapper around it that works fairly well. Eclipse has the Eclipse Memory Analyzer which is the one I use most often, just seems to handle large dump files a bit better. There's also a command line option, -XX:+HeapDumpOnOutOfMemoryError that will give you a file that is basically a snapshot of your process memory when your program crashed. You can use any of the above mentioned tools to look at it, it can really help a lot when diagnosing these sort of problems.
Depending on how hard the program is working, it may be a simple case of the JVM not knowing when a good time to garbage collect may be, you might also look into the parallel garbage collection options as well.
I also faced the same problem.I resolved by doing the build by following steps as.
-->Right click on the project select RunAs ->Run configurations
Select your project as BaseDirectory. In place of goals give eclipse:eclipse install
-->In the second tab give -Xmx1024m as VM arguments.
I would like to add that this problem is similar to common Java memory leaks.
When the JVM garbage collector is unable to clear the "waste" memory of your Java / Java EE application over time, OutOfMemoryError: Java heap space will be the outcome.
It is important to perform a proper diagnostic first:
Enable verbose:gc. This will allow you to understand the memory growing pattern over time.
Generate and analyze a JVM Heap Dump. This will allow you to understand your application memory footprint and pinpoint the source of the memory leak(s).
You can also use Java profilers and runtime memory leak analyzer such as Plumbr as well to help you with this task.
Try adding -Xmx for more memory ( java -Xmx1024M YourClass ), and don't forget to stop referencing variables you don't need any more (memory leaks).
Are you keeping references to variables that you no longer need (e.g. data from the previous simulations)? If so, you have a memory leak. You just need to find where that is happening and make sure that you remove the references to the variables when they are no longer needed (this would automatically happen if they go out of scope).
If you actually need all that data from previous simulations in memory, you need to increase the heap size or change your algorithm.
Java is supposed to clear the heap space for you when all of the objects are no longer referenced. It won't generally release it back to the OS though, it will keep that memory for it's own internal reuse. Maybe check to see if you have some arrays which are not being cleared or something.
No. The heap is cleared by the garbage collector whenever it feels like it. You can ask it to run (with System.gc()) but it is not guaranteed to run.
First try increasing the memory by setting -Xmx256m
I'm trying to find memory leaks and performance issues with my java application. Is there a program out there that can help me debug my application and display performance results?
Thanks.
Have a look at jvisualvm in the JDK - a subset of the Netbeans profiler - which can attach to a running Java 6 process and allow you to profile it and do memory analysis.
https://visualvm.dev.java.net/gettingstarted.html
I used a lot of tools to find why my program eats 100+ Mb of ram, polished the code to remove any possible memory leaks. Later I found that once jvm took some memory from the OS, I doesn't always return it, even if that memory is not used, which often looks like a memory leak. This depends on -Xmx and -XX:MaxHeapFreeRatio. I set Xmx to 40 which is roughly how much memory my app should use, and memory usage stays within 10-15 Mb of this range instead of increasing uncontrollably.
Also, jconsole is a great tool. It comes with jdk.
Eclipse has a good memory dump analyzer; but finding a memory leak can be very challenging and requires you to dive deeply into the way the objects are allocated by your application.
It took me 2 full days to figure out that one of my custom JTable cell editor classes was allocating a JDialog upon instantiation, without actually opening it, and the native part of the dialog kept the cell editor instance locked, thus the table, thus the screen and thus all entity objects that were associated with it.
You can try performance inspector tool.Following is the URL.
http://perfinsp.sourceforge.net/
Java Application performance is directly proportional to how JVM is running your application. This tool gives very good profiling information about JVM.But its not a graphical tool,you need to go through the text file generated. But its one time effort and you can get handy with this tool.I used it many time for performance related issues and it helped me lot.
We have been facing Out of Memory errors in our App server for sometime. We see the used heap size increasing gradually until finally it reaches the available heap in size. This happens every 3 weeks after which a server restart is needed to fix this.
Upon analysis of the heap dumps we find the problem to be objects used in JSPs.
Can JSP objects be the real cause of Appserver memory issues? How do we free up JSP objects (Objects which are being instantiated using usebean or other tags)?
We have a clustered Websphere appserver with 2 nodes and an IHS.
EDIT: The findings above are based on the heap-dump and nativestderr log analysis given below using the IBM support assistant
nativestd err log analysis:
alt text http://saregos.com/wp-content/uploads/2010/03/chart.jpg
Heap dump analysis:
![alt text][2]
Heap dump analysis showing the immediate dominators (2 levels up of hastable entry in the image above)
![alt text][3]
The last image shows that the immediate dominators are in fact objects being used in JSPs.
EDIT2: More info available at http://saregos.com/?p=43
I'd first attach a profile tool to tell you what these "Objects" are that are taking up all the memory.
Eclipse has TPTP,
or there is JProfiler
or JProbe.
Any of these should show the object heap creaping up and allow you to inspect it to see what is on the heap.
Then search the code base to find who is creating these.
Maybe you have a cache or tree/map object with elements in and you have only implemented the "equals()" method on these objects, and you need to implement "hashcode()".
This would then result in the map/cache/tree getting bigger and bigger till it falls over.
This is only a guess though.
JProfiler would be my first call
Javaworld has example screen shot of what is in memory...
(source: javaworld.com)
And a screen shot of object heap building up and being cleaned up (hence the saw edge)
(source: javaworld.com)
UPDATE *************************************************
Ok, I'd look at...
http://www-01.ibm.com/support/docview.wss?uid=swg1PK38940
Heap usage increases over time which leads to an OutOfMemory
condition. Analysis of a heapdump shows that the following
objects are taking up an increasing amount of space:
40,543,128 [304] 47 class
com/ibm/wsspi/rasdiag/DiagnosticConfigHome
40,539,056 [56] 2 java/util/Hashtable 0xa8089170
40,539,000 [2,064] 511 array of java/util/Hashtable$Entry
6,300,888 [40] 3 java/util/Hashtable$HashtableCacheHashEntry
Triggering the garbage collection manually doesn't solve your problem - it won't free resources that are still in use.
You should use a profiling tool (like jProfiler) to find your leaks. You problably use code that stores references in lists or maps that are not released during runtime - propably static references.
If you run under the Sun 6 JVM strongly consider to use the jvisualvm program in the JDK to get an inital overview of what actually goes on inside the program. The snapshot comparison is really good to help you get further in which objects sneak in.
If Sun 6 JVM is not an option, then investigate which profiling tools you have. Trials can get you really far.
It can be something as simple as gigantic character arrays underlying a substring you are collecting in a list, for e.g. housekeeping.
I suggest reading Effective Java, chapter 2. Following it, together with a profiler, will help you identify the places where your application produces memory leaks.
Freeing up memory isn't the way to solve extensive memory consumption. The extensive memory consumption may be a result of two things:
not properly written code - the solution is to write it properly, so that it does not consume more than is needed - Effective Java will help here.
the application simply needs this much memory. Then you should increase the VM memory using Xmx, Xms, XX:MaxHeapSize,...
There is no specific to free up objects allocated in JSPs, at least as far as I know. Rather than investigationg such options, I'd rather focus on finding the actual problem in your application codes and fix it.
Some hints that might help:
Check the scope of your beans. Aren't
you e.g. storing something user or
request specific into "application"
scope (by mistake)?
Check settings of web session timeout in your web application and
appserver settings.
You mentioned the heap consumption grows gradually. If it's indeed so,
try to see by how much the heap size
grows with various user scenarios:
Grab a heapdump, run a test, let the
session data timeout, grab another
dump, compare the two. That might
give you some idea where do the objects on heap come from
Check your beans for any obvious memory leaks, for sure :)
EDIT: Checking for unreleased static resources that Daniel mentions is another worthwhile thing :)
As I understand those top-level memory-eaters are cache storage and objects stored in it. Probably you should make sure that your cache is going to free objects when it takes too much memory. You may want to use weak-ref if you need cache for live objects only.
I'm not experienced with java applications but I found out that finding static pointers etc. to these applications' memory addresses is often (nearly) impossible, apparently because of the java engine that handles the code (correct me if this way of naming it is wrong please).
Now, I've used VisualVM (https://visualvm.dev.java.net/) and it's great. I can select my java process and create a heap dump. It then shows me all classes and their values.
Can I use this method to continousely poll the heap dump and receive object values, for example the X Y and Z of a game? How would I programmatically interact with such application, and if this should not be done with VisualVM, what would be an alternative?
Edit: this is what I need to do:
I need to be able to find all classes with properties that have a certain value. For example: I'd search for the X coordinate (a float) and it should return the class "PlayerCoordsHandler" (just an example) and the corresponding float with it's value... or alternatively just a way to find this same float again (after restarting for example). This process does not have to be programmatic, aslong as requesting the value of the now known property (x float) can be retrieved programmatically (for example with a command line utility or reading from a file).
Edit2:
The target application is a windows executable (but made with java) and launches it's own java VM. It's not possible to add java parameters for debugging. This does not seem to be required though, as VirtualVM is able to debug the process just fine. Anyone knows how?
Thanks in advance.
It looks like you want to debug running Java applications.
The "official" Java debugger is JDB. I believe it's part of the JDK. It has the ability to set breakpoints, examine heaps, list and display and even change variables, show running threads and so on. The usual debugger stuff. But it's command line, which makes it a pain in the neck to work with.
Instead, it makes a lot of sense to use an IDE with integrated debugger. I use Eclipse. You can do all the usual debuggery things, including displaying windows with variables. You can set conditional breakpoints and there's much more. Specifically in answer to your question, you can set up watch expressions, which will be evaluated during the program's execution and their displays refreshed with new values when they change.
You may not want to run your Java app inside the IDE; or it may be running in a Web application server. That's no problem for JDB or Eclipse (or other IDEs, like NetBeans or IntelliJ Idea): They can connect to a running JVM and debug remotely with the same level of convenience.
A program being debugged like this, remotely or otherwise, run somewhat more slowly than if it were not. Your game, while being debugged, will run at rather bad-looking FPS; but it should still respond more or less normally to gameplay interaction.
Remote debugging:
To be able to attach your EclipseNetBeans debugger to a running Java process you need to start that process with the following Java options…
-Xdebug -Xrunjdwp:transport=dt_socket,address=3704,server=y,suspend=n
Have a look at YourKit. You can monitor CPU, memory and threads live, and generate dumps whenever you want. It can even compare different memory dumps to show you which objects were added/removed.
It's not free though, it has a 15 day (or 30 day?) fully functional eval period. If free is not a real concern it's definitely a great tool.
I good starting point is the jps and jstat tools added in Java 6 (i think). jps gives you the pid and main class for each application. jstat give you more details about process
Triggering a heapdump is usefull for post-mortem analysis of say memory leaks, but as the Java garbage collector moves objects around, you cannot use the memory values of a heapdump to reliably access those objects.
If you need a way to query internal values from outside of the application you could look into setting up an RMI service API via which you can retrieve the values you need.
Another method (if you just need to test something) could be to connect to the process via de Java debugging API.
If you know the JRE location that is used, you could rename java.exe and write a (C/C++) wrapper that adds the debug options listed by Carl and calls the renamed_java.exe in turn.
Another posibility might be to add or update classes in the .jar file of the application. You do not need the source to do this.
Tom, are you trying to reverse engineer an application that specifically tries to obfuscate its working? If so you might get further if you contact the manufacturer and ask them what possibilities they see for what you try to achieve?
You can easily generate a heap dump by creating your own JMX connection to the JVM, just like VisualVM does it. Analyzing the heapdump is very possible (the data is there and totally disconnected from the JVM so there is no interference from the gc).
However, unless it is a very specific scenario you are looking for you are probably much better off giving the heapdump to MAT and find a good workflow in there to use.
Edit: In this particular case it is probably better to create some kind of specific API to access the values from the outside (and maybe publish the values as MBeans using JMX). Taking a heap dump is way to much work if all you want to do is monitoring a few values.
Edit2: Based on your edits, it seems to me like you could really benefit from publishing your own MBean over JMX. I have to run for a meeting but, unless someone else does it while I am away, I will try to remember to give you some pointers later. Either in an edit of this one or in a new post.
If you want to poll the values of specific objects while your Java application is running you would probably find that using JMX is a better and more efficient approach rather than using a heap dump. With JMX you can define what values should be exposed and use tools such as VisualVM or JConsole to view them at runtime.
With VisualVM and heapdump you can find all classes with certain property by OQL:
var out = "";
var cls = filter(heap.classes(), "/java./(it.name)")
while (cls.hasNext()) {
var cl = cls.next();
var fls = cl.fields;
while (fls.hasMoreElements()) {
var fl = fls.nextElement();
if (/size/(fl.name)) {
out = toHtml(cl) + "." + fl.name + "()\n";
}
}
}
out.toString()
and write custom logging for BTrace
It is alternative for debugging.
FusionReactor could be a good alternative. For example;
VisualVM doesn’t give you a lot of insides on application memory
except for the total Heap allocation. Heap is a good metric to start
with, but I feel this is not enough to troubleshoot the actual cause
of a memory-related issue.
FusionReactor will display all of the memory spaces it detects, which
depends on the version of Java you’re running:
Heap allocation Non-Heap allocation CodeHeap (profiled and
non-profiled methods) Compressed Class Space FusionReactor also shows
the amount of memory that each generation takes Eden Space Old Space
Survivor Space
https://www.fusion-reactor.com/blog/java-visualvm-alternatives/