I have encountered some strange text formatting in our application and I need to find the cause. The problem is I have no idea where to start looking in our Legacy code.Is it possible to make the IntelliJ IDEA debugger search for a value throughout the entire instance of an application in the JVM? As opposed to the usual way of setting a breakpoint with a condition.
Judging from the comment on this question, there is no such feature in IntelliJ.
You could however try performing a heap dump and then searching it using OQL. It can be done in the VisualVM tool which is bundled with the JDK:
Run the jvisualvm command (assuming you have the JDK's bin folder in your PATH).
Start your application, find it in the left column under Local, and select Heap Dump from the right-click menu.
Head to the OQL Console view. In Query Editor, type your query, for example:
select s from java.lang.String s where s.toString().contains("hello")
and execute it.
If your object is on the heap, you should see it. Click it and check the referencing objects in References.
This should get you closer to the classes/objects your object is used by.
There are some tricky parts:
you have to perform the heap dump before the object in question is garbage-collected - that is, as soon as possible after you are sure the object was created (other ways to acquire a heap dump are described here),
the object can be unreachable. In such cases, try to look through the object graph it is part of and find other objects which are still referenced,
the object may not live in the heap at all (escape analysis).
Another tool for heap dump analysis is Eclipse Memory Analyzer (MAT). (For your use case, enable the Keep unreachable objects option before analyzing the heap dump.)
You mentioned that you need to search for the value at runtime. However, if you suspect what you are looking for is a string literal, try just searching the JARs of your application as described in this answer.
Related
I have created an application using Vaadin having respective UI.
I am running in a server with having a maximum heapload of 250 Mb. The application gets crashed because of the heapload since it is not garbage collected.
I tried to run with a visualVM analyzer. Found to have lot of instances and somehow the vaadin ScssCache is making this mess.
How can I rectify this error? Is it because of the browser cache settings or should I do something with the vaadinservletcache entry?
I really do not understand please help. I have attached my VisualVm screen shot for the reference. Thank you very much. I am using vaadin 7.6.3.
The attached VisualVM screenshot shows that the entire scssCache retains 1248kb of memory, out of which 1200kb is used for the actual cached CSS contents. This is less than 1% of your 250mb heap size and most likely not the problem.
That 1200kb char[] with the compiled CSS might be the biggest individual object on the heap, but there's only one such object. You will thus have to look for something else that consumes lots of memory. I'd recommend looking at the list of Classes sorted by their retained size, and ignore low-level classes such as char[], java.lang.String or java.Util.HashMap and instead try to pinpoint anything related to your own application.
I would also encourage you to verify that your application is actually running in production mode since the only code path that I can identify that does anything with scssCache is through VaadinServlet.serveOnTheFlyCompiledScss which checks whether production mode is enabled and in that case returns before touching the cache.
Its not problem specific but is it possible to get a copy of current memory state, as in just get whatever is there in the main memory. I mean is there any way we can get an image of RAM in java.
I am editing my question. So here is a screenshot of my Windows 7 Task Manager.
#peter I see that current memory usage is 3.27GB. So, can I get that whole thing in some read only memory and when I restart my OS, it resumes where I left off, as in whatever my last memory snapshot was.
Yes, it's called a heap dump.
jmap -heap {pid}
dumps the heap to a file.
You can use jvisualvm to analyse the heap dump.
It depends on what you mean by "main memory".
The JRE is designed
to insulate your Java program from the OS around it, and vice versa.
to insulate Java objects from the implementation details of other objects
With "ordinary" Java, you only get to see what classes and objects expose through public methods.
However, Java has its Reflection API, and with that, if your JRE is configured to allow it, you can break through these boundaries, and look deeper into the classes and objects within the JRE.
In Oracle HotSpot, you can start the JRE with the Java Servicability Agent - http://openjdk.java.net/groups/hotspot/docs/Serviceability.html - this gives you access through an API to much more detail of the Java heap. But you're still restricted to memory claimed by the JRE process, and allocated to its heap.
One further possibility is to write a native library using JNI. There are C API calls that allow you to browse the OS address space. You need to be root (or the equivalent on your OS) to see other people's address space). You could write C code, and JNI to call it from Java.
I have a java project written in eclipse (RAD, actually); it uses a significant amount of memory by virtue of using iText. I am looking at using a different way of generating my iText document that is supposed to use less memory. I want to know how much less memory it uses.
I know what object will be the root for the largest portion of the memory; it would be fine for my purposes if I could set a breakpoint and then do something that would tell me the deep-copy memory used starting with that object (i.e., the memory used by it and all of its direct and indirect references).
I've been looking at memory monitors and heap dump analyzers and so forth for an hour now, and am pretty confused. All of them appear to be pointed at answering a different problem, or at least pointed to such a general class of problems that, even if I could get them installed and working, it is not clear whether they would answer MY question.
Can someone point me to a reasonably simple way to answer this limited question? I figure if I run the code once the current way, find out the memory used by this object and maybe one or two others, then run it again and look at the same values, I'll know how much good the new technique is doing.
rc
http://www.eclipse.org/mat/
works great for me... tutorials are included
You can fire up JVisualVM, shipped with all Oracle JDK and available independently.
Monitor your process, and at some point, you can do a Heap Dump.
In the heapdump tab, you can go to the OQL console and select your object[s].
When viewing the instance of an object, you can request to compute the retained size. That will give you the total size of your object.
How come every object appears to be marked new, instead of just objects that are in the second snapshot but not in my baseline snapshot? Looking around online, I see some suggestions that I need to use hprof instead of jmap to make my memory dumps, but it appears that hprof generates dumps in exactly the same format.
This is JDK 1.6.0_14; I have tried on both Windows and UNIX.
jhat -baseline indeed won't work with dumps produced by jmap. I'm not certain, but I believe this is because hprof attaches to the JVM right from the start and keeps its own track of objects, allowing it to produce consistent IDs across multiple dumps. Don't quote me on that. Either way, the important point as far as you're concerned is that jmap dumps don't work.
However, all is not lost. Go and get the Eclipse Memory Analyzer. (If you don't use Eclipse, fear not, you can get it as a standalone executable.) It's faster than jhat, uses less memory than jhat and it can do what you want:
Open dump2 (with File|Open Heap Dump). Don't bother having it create a report for you.
Open dump1 (same way). Again, no report.
In the tab for dump2, click "Histogram"
On the right of the toolbar in the Histogram subtab is "Compare to another Heap Dump". Click it.
Select dump1 from your dialog as the dump to use as the baseline.
Presto, you have the differences between dump2 and the baseline dump1.
All of this works fine with a jmap dump.
It seems that you need to use hprof. But are you sure you use the same VM instance ?
The -baseline option allows two dumps to be compared if they were produced by HPROF and from the same VM instance. If the same object appears in both dumps it will be excluded from the list of new objects reported. One dump is specified as a baseline and the analysis can focus on the objects that are created in the second dump since the baseline was obtained.
If everything is considered as new, I will make sure that's the same instance of the VM.
Here
I'm not experienced with java applications but I found out that finding static pointers etc. to these applications' memory addresses is often (nearly) impossible, apparently because of the java engine that handles the code (correct me if this way of naming it is wrong please).
Now, I've used VisualVM (https://visualvm.dev.java.net/) and it's great. I can select my java process and create a heap dump. It then shows me all classes and their values.
Can I use this method to continousely poll the heap dump and receive object values, for example the X Y and Z of a game? How would I programmatically interact with such application, and if this should not be done with VisualVM, what would be an alternative?
Edit: this is what I need to do:
I need to be able to find all classes with properties that have a certain value. For example: I'd search for the X coordinate (a float) and it should return the class "PlayerCoordsHandler" (just an example) and the corresponding float with it's value... or alternatively just a way to find this same float again (after restarting for example). This process does not have to be programmatic, aslong as requesting the value of the now known property (x float) can be retrieved programmatically (for example with a command line utility or reading from a file).
Edit2:
The target application is a windows executable (but made with java) and launches it's own java VM. It's not possible to add java parameters for debugging. This does not seem to be required though, as VirtualVM is able to debug the process just fine. Anyone knows how?
Thanks in advance.
It looks like you want to debug running Java applications.
The "official" Java debugger is JDB. I believe it's part of the JDK. It has the ability to set breakpoints, examine heaps, list and display and even change variables, show running threads and so on. The usual debugger stuff. But it's command line, which makes it a pain in the neck to work with.
Instead, it makes a lot of sense to use an IDE with integrated debugger. I use Eclipse. You can do all the usual debuggery things, including displaying windows with variables. You can set conditional breakpoints and there's much more. Specifically in answer to your question, you can set up watch expressions, which will be evaluated during the program's execution and their displays refreshed with new values when they change.
You may not want to run your Java app inside the IDE; or it may be running in a Web application server. That's no problem for JDB or Eclipse (or other IDEs, like NetBeans or IntelliJ Idea): They can connect to a running JVM and debug remotely with the same level of convenience.
A program being debugged like this, remotely or otherwise, run somewhat more slowly than if it were not. Your game, while being debugged, will run at rather bad-looking FPS; but it should still respond more or less normally to gameplay interaction.
Remote debugging:
To be able to attach your EclipseNetBeans debugger to a running Java process you need to start that process with the following Java options…
-Xdebug -Xrunjdwp:transport=dt_socket,address=3704,server=y,suspend=n
Have a look at YourKit. You can monitor CPU, memory and threads live, and generate dumps whenever you want. It can even compare different memory dumps to show you which objects were added/removed.
It's not free though, it has a 15 day (or 30 day?) fully functional eval period. If free is not a real concern it's definitely a great tool.
I good starting point is the jps and jstat tools added in Java 6 (i think). jps gives you the pid and main class for each application. jstat give you more details about process
Triggering a heapdump is usefull for post-mortem analysis of say memory leaks, but as the Java garbage collector moves objects around, you cannot use the memory values of a heapdump to reliably access those objects.
If you need a way to query internal values from outside of the application you could look into setting up an RMI service API via which you can retrieve the values you need.
Another method (if you just need to test something) could be to connect to the process via de Java debugging API.
If you know the JRE location that is used, you could rename java.exe and write a (C/C++) wrapper that adds the debug options listed by Carl and calls the renamed_java.exe in turn.
Another posibility might be to add or update classes in the .jar file of the application. You do not need the source to do this.
Tom, are you trying to reverse engineer an application that specifically tries to obfuscate its working? If so you might get further if you contact the manufacturer and ask them what possibilities they see for what you try to achieve?
You can easily generate a heap dump by creating your own JMX connection to the JVM, just like VisualVM does it. Analyzing the heapdump is very possible (the data is there and totally disconnected from the JVM so there is no interference from the gc).
However, unless it is a very specific scenario you are looking for you are probably much better off giving the heapdump to MAT and find a good workflow in there to use.
Edit: In this particular case it is probably better to create some kind of specific API to access the values from the outside (and maybe publish the values as MBeans using JMX). Taking a heap dump is way to much work if all you want to do is monitoring a few values.
Edit2: Based on your edits, it seems to me like you could really benefit from publishing your own MBean over JMX. I have to run for a meeting but, unless someone else does it while I am away, I will try to remember to give you some pointers later. Either in an edit of this one or in a new post.
If you want to poll the values of specific objects while your Java application is running you would probably find that using JMX is a better and more efficient approach rather than using a heap dump. With JMX you can define what values should be exposed and use tools such as VisualVM or JConsole to view them at runtime.
With VisualVM and heapdump you can find all classes with certain property by OQL:
var out = "";
var cls = filter(heap.classes(), "/java./(it.name)")
while (cls.hasNext()) {
var cl = cls.next();
var fls = cl.fields;
while (fls.hasMoreElements()) {
var fl = fls.nextElement();
if (/size/(fl.name)) {
out = toHtml(cl) + "." + fl.name + "()\n";
}
}
}
out.toString()
and write custom logging for BTrace
It is alternative for debugging.
FusionReactor could be a good alternative. For example;
VisualVM doesn’t give you a lot of insides on application memory
except for the total Heap allocation. Heap is a good metric to start
with, but I feel this is not enough to troubleshoot the actual cause
of a memory-related issue.
FusionReactor will display all of the memory spaces it detects, which
depends on the version of Java you’re running:
Heap allocation Non-Heap allocation CodeHeap (profiled and
non-profiled methods) Compressed Class Space FusionReactor also shows
the amount of memory that each generation takes Eden Space Old Space
Survivor Space
https://www.fusion-reactor.com/blog/java-visualvm-alternatives/